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ABSTRACT 

Low-inference measures of teacher process variables 
from two behavioral observation systems were taken on a sample of 31 
teachers selected because of their consistency in producing student 
learning gains on the Metropolitan Achievement Test and were 
correlated with student outcome measures. Correlations show the 
strength of relationships with success in producing student gains. 
Findings from the first year of a 2-year study attempt to isolate 
correlates of effective teaching. (Appendixes describing coding 
methods are included.) (Author) 
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Abstract 

Low- inference measures of teacher process variables from two behavioral 
observation systems were taken on a sample of 31 teachers selected because of 
their consistency in producing student learning gains on the Metropolitan 
Achievement Test and were correlated with student outcome measures. Correlations 
showing the strength of relationships with success In producing student gains 
are presented. Data represent findings from the first year of a two-year study 
attempting to Isolate correlates of effective teaching. 



Low- Inference Observational Cod!ng Measures 
and Teacher Effectiveness 

Tjntll recently, teacher effectiveness research attempting to link teacher 
behavior In the classroom to student achievement outcome measures produced gener- 
ally disappointing results. Reviews of literature (Mitzel and Gross, 1958; 
Morsh and Wilder, 1954) unhappily concluded that efforts In this area had not 
led to the Identification of specific Teacher behavior which was reliably linked 
to student achievement gains. 

These results led many to conclude that teaching Is an art rather than a 
science, and that attempts to Identify universally effective teachinq behavior 
or to Identify "effective teachers" were fruitless. However, more recent re- 
views (Flanders and Simon, 1969; Rosenshlne and Furst, 1971; Rosenshlne, 1971; 
Dunkin and BIddle, 1973) provide greater cause for optimism, concluding that 
teaching behaviors related to teachers' general effectiveness are being Identi- 
fied more consistently In recent studies. Although the posHive results usually 
involve relatively weak correlations between teacher process measures and stu- 
dent outcome variables, several studies have agreed In identifying certain teach- 
er variables as aspects of general effectiveness. 

One reason has been the use of better observation systems. Much early re- 
search used systems developed by psychologists to study group dynamics. Gener- 
ally, such systems are not very appropriate for teacher effectiveness research. 
They were not developed for use In the classroom, and usually were not constructed 
specif ical I / to determine whether a teacher met specified objectives, either In 
his teaching behavior or In the learning of his students. 

Thus one reason that process-product research In teacher effectiveness has 
Improved of late has been the appearance of new systems and Improvements In some 
of the older systems (Flanders, I970)t 
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Despite these improvements In mei-hodology and other signs of progress in 
this line of research, a new threat to the search for effective teaching be- 
havior arose with the publication of Rosenshlne's (1970) review of stability 
across time In teachers' abllHIes to produce student learning gains. After 
reviewing a large body of literature, Rosenshlne could locate only five studies 
which contained data on stability in teacher effectiveness over long time per- 
iods in producing student learning gains. Of the five, only two seem immediately 
general izab I e to the typical school situation. One Involved Air Force Instruc- 
tors teaching eight hour airplane hydraulics courses to Air Force recruits, and 
two of the others involved experimental studies where teachers were not using 
their typical methods of Instruction. Thus only two of the five studies in- 
volved ordinary school teachers teaching in their normal ways. One of these 
studies did not give an exact stability coefficient but suggested that stabili- 
ty from one year to the next was quite low, while the stability coefficient from 
the other study was .09. These figures quite obviously suggest that teacher 
effectiveness in producing student learning gains Is not a stable "trait," that 
a teacher who produces large gains In his students this year is not necessarily 
going to do the same the next year. Such results. If they accurately reflect 
the general case, render the Improvements in classroom observation methodology 
Insignificant, since there Is little point In process-product teacher effective- 
ness research If teachers are not ttable from one year to the next in their rela- 
tive success In producing student learning gains. 

The present study 

The data to be presented are part of a large scale study which attempts to 
address Itself to several of the problems described above. The study Includes 
several methodological innovations specifically designed to overcome some of 



the difficulties in earlier research. It has Involved two major parts to date. 
The first, a study of stability In teacher effectiveness in producing student 
learning gains, established that such stability Is much more evident than the 
two studies mentioned above would suggest, at least in certain teachers (Brophy, 
1972). The second part of the study is a long term and multifaceted attempt 
to Identify the personal and behavioral correlates of teaching effectiveness 
In a sample of teachers known to be stable In their relative effectiveness In 
producing student learning gains. 

Stability In teacher effectiveness 

The stability aspect of the study Involved 88 second grade teachers and 77 
third grade teachers from an urban school district. Study of stability was pre- 
dicated upon the following two basic assumptions: 

1. In evaluating the Impact of schooling on student learning, the teacher, 
and not the school. Is 'he appropriate unit of analysis. This point would seem 
to be Intuitively obvious. However, the study that formed the basis for the 
original Coleman Report (Coleman, et aL, 1966), as well as several studies 
don e since (notably Jencks, et al., 1972), have used schools rather than teach- 
ers as the unit of analysis. This method is indirect at best. Although one 
school might be more effective than another because It contains a greater per- 
centage of highly effective t^ache^s, the teachers, and not the school, are the 
effective causal mechanisms producing student learning gains. This would be 
merely a minor technical point except for the fact that such studies have re- 
cently been used as the basis for claims that schools do not have differential 
effects. This In turn leads to the conclusion that a school *s effectiveness Is 
determined by the socioeconomic status of Its student population rather than by 
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the skills of Its teaching staff, and the latter at least Implies that schools 
should not be held accountable for or expected to accomplish much with students 
from low socioeconomic backgrounds. 

The latter conclusions simply do not follow from the data. The studies 
used to "support" them havtt failed to directly measure teaching effectiveness. 
They have used only Indirect presage measures (teacher examination scores, 
average years of experience of the teaching staff, percentage with advanced 
degrees, student/teacher ratlo^ per pupil expenditure, etc.). None have In- 
cluded process observations of teacher behavior, even though the Indirect Indi- 
ces mentioned above either are already known to be unrelated to teacher effec- 
tiveness or have never been studied In relation to It. If we assume that teachers 
of varying effectiveness are randomly distributed throughout a school system, or, 
as Is more probable, that the better teachers are more often assigned to the 
higher socioeconomic schools. It is hardly surprising that studies using the 
chool as the unit of analysis come out with the kinds of results that they do. 
To conclude from such data that schools (and, by Implication, teachers) have no 
effect Is patently fallacious. It Is more than a merely harmless error, how- 
ever; It appears to have already resulted In cutbacks In funding for schools 
and for educational research In some quarters. As reanalyses of the Coleman 
data showed (Mood, 1970), and as data from the present study support, teachers 
are differentially effective and do make a difference (Veldman and Brophy, 1973). 

2. In addition to using the teacher as the unit of analysis, studies of 
stability In teacher effectiveness must Include the most appropriate types of 
teachers If they are to be optimally useful. Many of the studies previously 
done. Including some of those reviewed by Rosenshfne, Involved teachers who were 
working In a special experimental program. It is reasonable to suppose that 



such teachers are less stable In their teaching behavior than teachers working 
naturallstfcal ly In typical and familiar settings. 

In addition, several other studies have Included, or even have been con- 
fined to, student teachers or teachers In their first year or two of teaching 
experience. The classroom behavior of such teachers Is known to be unstable; 
they are In the process of learning how to operate In the classroom and have not 
yet established a stable teaching pattern or "style." Thus Inclusion of teach- 
ers In a study of stability In teacher effectiveness Is Inappropriate If th> 
intention Is to generalize the data to the average or typical career teacher. 

Teacher selecl on 

Consistent with the two preceding assumptions. It was decided to restrict 
the study to teachers who had been teaching at the same grade for at least 
three consecutive years, avoiding student teachers and brand new teachers, and 
to use the teacher rather than the school as the unit of analysis. Resource 
limitations demanded that the study also be restricted to only two grade levels. 
Partly out of an Interest in early education, and partly on the assumption that 
teachers probably make a greater difference in the learning of younger students 
than older ones, because the younger students are less capable of overcoming the 
effects of Inadequate teaching through their own learning efforts than olcer stu- 
dents, the decision was made to work at the early elementary grades. The first 
grade was rejected for lack of an adequate pretest. The children did take readi- 
ness tests at the beginning of first grade, but these are known to be unreliable 
and heavily influenced by the child's preschool experience, especially the stimu- 
lation he receives at home (Hess, 1970). Thur^ the second and third grades were 
selected for study. 
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The school district contained 50 elementary schools with about 275 teachers 
working at grad. 2 and 3, Ot these, 88 second grade teachers and 77 third grade 
teachers met the selection criteria mentioned above. The district administered 
the Met ropo 1 1 tan- Achievement Tests each fall. Residual gain scores from 3 
language arts and 2 math subtests were used for determining teacher effectiveness. 
Data from Title I and non-Title I schools were treated separately, because 
si Ightly different forms of the MAT were used In these two types of schools. 
Second and third grade data were also treated separately, yielding a total of 
4 separate data sets. 

For each data set residual gain scores on each subtest were computed for 
each student within sex and within each of the three years, using the student's 
prescore as a covarlate. Student residual gain scores were then collated by 
classroom, and a mean residual gain score N^as computed for each teacher for each 
subtest for each of the three years Included In the study, Irtercorrelatlons 
among these mean residual gain scores were then computed. 

The data show generally high correlations across subtests within years, 
and moderately high stability within subtests across years (Brophy, 1972), 
Within years, the three language arts subtests (Word Knowledge, Word Discrimi- 
nation, and Reading) correlate highly with one another, and usually also cor- 
relate highly with arithmetic reasoning. Arithmetic computation typically cor- 
relates highly only with arithmetic reasoning. The fact that the correlations 
across subtests within years are quite high and generally higher than the sta- 
bl llty coefficients for single subtests across yfiars Illustrates that even when 
statistical procedures are used to adjust scores for student performance on 
tne pretest and for other variables, a yearly class or cohort effect exists. 
This may represent unmeasured class coheslveness, motivational factors, student 



or teacher absentee I sm^ or other factors that operate within but not across 
years. This general yoarjy effect suggested by the high correlations across 
subtests within years was verified In an Informal cnalys?s of each teacher's 
set of residual gain scores. Only about 15 of the teachers s^^owed a consistent 
tendency to produce higher resloual s^'" scores In language arts than In math 
or vice versa. 

Despite this yearh cohort effect the stabllHy coefficients reflecting 
teacher consistency In producing student learning gains were notably higher 
than those reported In the 5 long-term studies reviewed by Rosenshlne (1970). 
While not high enough to Justify their use as teacher accojnvabi I !_ty criteria^ 
they were high enough to enable us to Identify consistent teachers tor further 
study. 

Stability coefficients were mostly Insignificant tn one of the data sets, 
but mostly signlrlcant In vhe other three. In the first set (N's«22-26), they 
ranged from -.12 to .49, with a median of .25; In the second set (N's=36-42), 
they ranged from .33 to .63, with a median of .42; In the third set 
(N»s==20-24), they ranged from .19 to .78, with a median of .39; and In the 
last set (N»s=42-44), they ranged from --07 to .65, with a median of .40 
(Brophy, 1972). About half of the 3-year patterns were linear, and the' remain- 
der were non-linear. Slightly more than half of the linear pttterns were con- 
stant, while the others «ihowed patterns of either Improvement or deterioration. 
Thus, when the sample Is restricted to teachers who have had several years o^ 
experience at the same grade level, the stability coefficients obtained are 
higher than those previously reported. 

Analyses of teachers* patterns across subtests and years revealed addition- 
al Information. Ff^st, most teachers wdr^ relatlv^My equally successful with 
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boys and girls; only four of 165 consistently produced higher residual gains 
in one sex than the other (although girls generally outgained boys, as usual). 
Thus student sex did not significantly affect teacher effectiveness. 

Also, no evidence of schools' effects could be found. High- and low-effec- 
tfve teachers were not concentrated in certain schools. There also was no 
evidence of a "rebound" effect within schools (a combination of low-effective 
second grade teachers with high-effective third grade teachers, or vice versa, 
which might mean that the data on the third grade teachers were artif actual ) . 
These data reinforce the point made earlier that teachers, and not schools, are 
the appropriate units. 

Finally, many teachers showed constancy across subtests within years as 
well as within subtests across years, so that teachers who produced generally 
consistent guins across subtests and across the two sexes could be identified. 
The thirty-one teachers included in the process observation study the first 
year were selected from this consistent group. 

Studying consistent teachers 

The second step In the research involved Intensive study of the classroom 
behavior of these generally consistent teachers. Since such teachers have al- 
ready demonstrated consistency in their ability to produce student achievement 
gains, and since they have all had several consecutive years of experience 
teaching at the same grade level and have probably, therefore, attained a stable 
pattern or "style," process-product research designed to identify the behavior- 
al correlates of teaching effectiveness seemed to be particularly promising on 
such a sample. Naturalistic observations in these teachers' classrooms were 
therefore undertaken. At this writing, one year of naturalistic research has 
already been conducted, and a second year Is presently completed. 



The first year's research was conducted on a sample of 31 teachers divided 
roughly evenly between Title I and non-Title I schools and between grade 2 and 
grade 3. The data collected during this year included process measures of 
classroom behavior and personality and attitude data from pencil and paper tests. 
The process measures Included both low-inference behavioral coding and high«in- 
f ere nee ratings by classroom observers. The present paper presents the findings 
from the two low-Inference systems used. Each teacher was observed for two 
mornings and two afternoons during the spring semester, totalling about eight 
hours of observation. The main low-inference coding was based on the Brophy- 
Good Dyadic Interaction System (Brophy and Good, 1970), although the original 
system was expanded to include certain additional variables, particularly class- 
room management variables based on Kounln's (1970) classroom management research. 
This system picked up such variables as teacher vs. student-initiated interac- 
tions, types of interactions (academic, procedural, or behavioral-disciplinary), 
difficulty level of teacher questions, quality of student responses to those 
questions, quantity and quality of teacher feedback and evaluative reactions to 
student responses and student work, and the teacher's methods and general effect- 
iveness in handling classroom management and disciplinary problems. 

This system was selected (and adapted) for use as the main low- inference 
data collection instrument because it subsumes most of the information Included 
in other major systems (although In slightly different form) and Includes several 
features vhlch do not appear In other systems, but are believed to be Important 
aspects of teaching in the primary grades. 

A second behavioral coding instrument, developed by staff member Itency 
Moore, was used on a subsample of 5 high-effective and 5 low-effective teachers 
who were observed twice during mornings when they did most of their group In- 
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structlon. This instrument was specially constructed to get at group Instruc- 
tion methodological variables, such as lesson composition, sequence, and 
clarity; teacher questioning patterns; and handling of seatwork assignments. 
This system was included to enable low-Inference coding of certain group in- 
structional behaviors not included in the other system. 

Both of these instruments involve low-inference coding of discrete class- 
room behaviors. Data were also collected with a penci l-and-paper test battery 
{Peck and Veldman, 1973) and with a number of high- inference rating scales and 
checklists (Evertson and Brophy, 1973). 

Observer training and reliability 

Observers using the expanded Brophy-Good system were trained according to 
the sequence outlined in Brophy and Good (1970), which Includes reading the 
manual, responding to questions about coding decisions, coding videotapes, and 
coding classrooms similar to those in the study. Each observer continued train- 
ing uttil he reached an 80% agreement criterion, using a strict definition In 
which % agreement = number of coding decisions made by both coders and agreed 
upon divided by Itself + number of coding decisions not agreed upon + number 
of codings made by the first coder but not the second + number of codings made 
by the second coder but not the first. 

This is a more stringent agreement criterion than is typically used; how- 
ever. It Is not impossible to attain In even a comprehensive low-inference cod- 
ing system, and the probable societal Implications of data from this research 
make it imperative that coding be as accurate as possible. Observers know the 
nature of the research bur had no Information concerning teachers* effectiveness 
scores, so that their coding could not be biased by this factor. 
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Even though ccxJers reached a high level of reliability during training, 
they worked in pairs throughout the study as an additional safeguard against 
coder bias. Differences between the two coders were resolved by averaging in 
computing teachers' scores for each observation. Agreement data for each coding 
decision made by coders using this system are given in Table L Note that many 
of these are lower than the 80% level reached in training, although most are 
satisfactorily high, given the strictness of the criterion. Most disagreements 
occurred because one coder missed a codable behavior durfng a rapidly paced se- 
quence (not because both coders coded but disagreed). This coder agreement was 
generally quite satisfactory, and the disagreement which did exist was due largely 
to the difficulty of "catching everything" during bursts of activity rather than 
to differences in the application of category definitions and/or bias toward 
or against teachers, 

RESULTS 

Expanded Brophy-Good System (See Appendix A) 

The results from the process variables in the expanded Brophy-Good System 
are presented in "^--"ble 2. The table contains correlations between 141 process 
measures (subdivided into 17 clusters) and mean residual gain scores for the 
five MAT subtests. Correlations are presented separately for interactions which 
occurred during the morning in whole class activities, interactions that occurred 
during the afternoons In whole class activities, and interactions that occurred 
during reading groups. This allows some indication of the degree to which the 
relationships generalize across three different classroom settings. 

Also, three coefficients are presented for each relationship. The top co- 
efficient is for the entire group of teachers (all teachers for whom scores were 
available), while the bottom two correlations are for the teachers in Title I 
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schools (on the left) and In non-Title I schools (on the right). Probability 
values are Indicated by underlining the coefficient once when probability is 
between .10 and .05, and underlining it twice when probability is below .05. 
These relatively lenient probability values were selected in place of the more 
typical .05 and .01 levels in view of the small N. Maximum N's are 31 for the 
entire sample, 13 for Title I, and 18 for non-Title I. The actual iVs are some- 
times lower, particularly in clusters E, F, G, J, L, and M. These clusters deal 
with aspects of classroom interaction that were rarely or even never observed in 
some classrooms. The findings will be discussed on a cluster-by-cluster basis. 

Cluster A concerns the teacher's method of selecting respondents to ques- 
tions. As expected, a teacher tendency to preselect the respondent before ask- 
ing the question was negatively correlated with achievement gains. This con- 
firms the typical advice given to teachers that better attention to questions will 
result if the teachers ask the question first (and presumably allow some time for 
thinking about the answer) before selecting a respondent. The data regarding 
cal ling on volunteers are the first of many to be discussed in the paper which 
show a difference in effectiveness in Title I compared with non-Title I schools. 
Teachers in Title I schools who called on volunteers tended to be more successful 
In producing student learning gains, but In non-Title I schools this behavior was 
negatively correlated with effectiveness. Thus teachers working in Title I schools 
need to "go after" students more systematically and proactively to qet them to 
respond in public response opportunity situations, and not merely confine them- 
selves to calling upon students who raise their hands. The most probable reason 
for this difference is a difference In student hand-raising response rates. Most 
students In non-Title I schools are eager to respond and raise their hands fre- 
quently, so that the teacher will be spreading around response opportunities even 
If she calls primarily upon those who have their hands raised* However, a teacher 
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^ who behaves this way in a Title I school is- likely to end up calling on the 

I same few students for the majority of the public response opportunities. To 

prevent this domination of public response opportunities by the brightest and/or 
best motivated students, teachers In Title I schools need to frequently call 
I - upon non-volunteers. 

' The data concerning students calling out answers are mixed, with the only 

significant relationship being a negative one. The most likely explanation is 
that this reflects classroom control: teachers probably have poor control in 

I classrooms where calling out is especially frequent. This will be checked out 

. in subsequent analyses in which teacher process variables will be correlated 

' with one another. 

j' The data of cluster B concerning the difficulty level of the questions that 

the teachers asked did not fall into any easily interpretab le pattern. This is 
I most probably because at second and third grade the vast majority of questions 

are of the product and choice variety, with very few process questions. Level 
^- of questions is probably a more Important variable at higher grade levels. The 

|' significant relationships that did exist were all in the Title I schools, but 

they were mixed in direction. The data for general class discussions show nega- 
1^ five relationships for process questions and positive ones for choice questions, 

suggesting that In these settings teachers in Title I schools need to take care 
to keep the discussion at a level that the students can understand and deal with. 
The data for reading ^roup questions are reversed, however. Here the majority 
of the process questions dealt with comprehension of the story beinn read or 
with word attack skills. Apparently the questions in this context were well 
within the grasp of the students in the Title I schools, so that they benefited 
by the stimulation of process questions, and were less stimulated by choice 
questions which may have boen too easy for them. 

The data in Cluster C concerning the quality of the children's answers to 
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questions provide several interesting findings. First, they contradict the advice 
of errorless learning advocates and support the idea that students should be 
challenged with questions at the threshold of their knowledge. This was especially 
true for students in non-Title I schools, where the percentage of correct answers 
regularly was negatively correlated witn student learning gains. In short, teach- 
ers should challenge bright and wel I -motivated students and not confine themselves 
to questions that the children can answer with ease. This generalization does 
not apply to Title I schools, however, where the pattern of relationships between 
student learning gains and the frequency of correct and Incorrect answers was 
more mixed. In other words, there is a greater danger here of producing unde- 
sirable effects by asking questions that are too difficult, especially during 
whole-class discussions (vs. reading groups). It had been expected that teach- 
ers whose students actively said, "I don^t know," when they did not know an an- 
swer would be more successful than teachers whose students made no response when 
they did not know an answer, but this was not the case. The tendency to say "I 
don't know" was positively correlated with student learning gains, but, with one 
exception, failure to respond also positively correlated, at least in Title I 
schools. This may mean that teachers should frequently ask questions, that they 
should ask difficult and challenging questions (especially In non-Title I schools), 
or both. 

When viewed in a somewhat different context, the data for Cluster C suggest 
that positive expectations are an important component of teaching effectiveness. 
That is, the effective teachers appear to be those who challenge the children 
with difficult questions and who are undeterred by incorrect answers, "don't 
know," or failure to respond. 

The data in Clusters D through H, dealing with teacher reactions to student 
answers or failure to answer, will be discussed in combination. First, the data 



on teacher praise flatly contradict the advice qlven in virtually all teacher 
training books and materials. Praise had relatively few significant correlations 
with student learning gains, and the correlations which were significant were 
all negative . Thus frequent teacher praise seems to be unimportant as a motivat- 
ing incentive, and overly frequent praise appears to actually interfere wi+h 
learning progress. This appears to be true for both Title I and non-Title I 
schools. Furthermore, the data do not reflect a satiation effect; praise was 
infrequent, even followinq correct answers. In short, despite the near-universal 
stress placed upon the Importance of praising children's responses, praise 
correlated negatively with student learning gains. 

Criticism for callinci out (correct) answers correlated positively in non- 
Title I schools but negatively In Title I schools. Like the data in Cluster A, 
this probably represents the difference in student eagerness to respond. If the 
students in non-Title I schools are all eager to respond, it may be important 
for teachers to insist that the class wait for her to call on someone and respect 
that student's right to answer the question without being Interrupted. In contrast, 
if students in Title I schools are generally unresponsive, allowing or even en- 
couraging calling out answers might be one way for the teacher to improve moti- 
vation. 

With two exceptions, all significant correlations involving teacher failure 
to give feedback were negative. This underscores the imporatnce of teachers 
giving feedback to students to tell them whether or not their answers were correct. 
The exceptions occurred for two measures of teacher reactions to "don't know" or 
no response in the Title I schools. Although It cannot be determined from the 
data, it is likely that the vast majority of the instances occurred following no 
response, in situations in which the teacher was moving at a brisk pace and pri- 
marily concerned with eliciting the answer rather than with dealing with an In- 
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dividual student at length. In these situations (rapidly paced drills or re- 
views, for example), it sometimes makes sense for the teacher to move on to 
someone else when a child does not respond, in order to keep the lesson moving 
at an appropriate pace. Thus failure to give feedback to students in such 
situations is understandable. Nevertheless, we were surprised that these re- 
lationships were strong enough to produce a significant positive correlation. If 
It is true that failure to give feedback does occur mostly during rapidly paced 
question sequences, it is possible that this finding involves a "proxy variable.'' 
That Is, perhaps the real relationship Is not between failure to give feedback 
and student learning gains but between rapidly paced question sequences and student 
Teaming gains. 

Although less surprising than the data on praise, the data on criticism of 
student responses also did not conform to our expectations. . We thought that cri- 
ticism of student error or failure to respond would be negatively correlated with 
student learning gains, but the results were mixed. Criticism simply was not an 
important variable In the Title I schools. Where it never correlated significantly 
with student learning gains, either positively or negatively. In contrast, the 
data for non-Title I schools showed several significant relationships, but they 
were about equally divided between positive and negative correlations. Our best 
guess is that the positive correlations represent justified teacher criticism of 
poor responses that resulted from inattention or misbehavior, while the negative 
relationships represented unjustified critical overreactlon by the teachers. 
However, the available data do not allow a test of this Interpretation. 

The frequency with which the teacher gives process feedback (giving a de- 
tailed explanation rather than merely supplying the correct answer) showed many 
significant correlations with student learning gains, all but one positive. This 
underscores the Importance of teachers* making an effort to be sure the children 
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truly understand the material, and this frequently will mean giving a detailed 
explanation or demonstration rather than just supplying the correct answer. The 
only significant negative relationship occurred In the reading groups in Title I 
schoois. This may tie in with the data in cluster B on question difficulty. Take 
together, the data reviewed so far suggest that Title I students have difficulty 
In general class discussions, where it might be Important for the teacher to err 
on the side of "over teaching," but that they have little difficulty with the 
kinds of questions, asked during reading groups, so that "over teaching" Is a lia- 
bility rather than an asse*^ in this context. 

The remaining teacher reaction variables all concern the question of whether 
the teacher provides the answer or allows another child to provide It when the 
student has not been able to respond correctly, or whether Instead she pursues 
the issue by repeating the question, rephrasing or giving a clue, or asking a 
new question. The frequency with which the teacher gives the answer herself cor- 
related mostly positively with student learning gains, while the frequency with 
which teachers called on someone else or allowed other students to call out the 
answer correlated negatively. These data suggest that If the teacher Is going to 
supply the answer to the student, she should do so herself (thereby remaining with 
the student and bringing the Interaction to a positive closure) rather than simply 
move on to another student or al low another student to call out the answer. 

As an alternative to supplying the answer, the teacher can stay with the 
student and attempt to get the answer from him or ask him a new question. The 
data on these variables are mixed. Concerning asking a new question following 
an initial correct response, most of the significant correlations are from Title I 
schools. They are negative in direction for general class discussion periods 
but positive for reading groups. The negative correlations tie in with our 
earlier interpretation that It Is especially Important in Title I schools to 



spread public response opportunities around the class rather than confine them 
to a few Individuals* However, this apparently is. not true for reading groups. 
The positive correlations here might simply represent a questioning pattern (such 
as asking an easy question first and then following up with a more difficult 
question), or it may represent a more psychologically meaningful relationship be- 
tween staying with a student following a correct answer and learning gains in 
a reading group context. These positive correlations In reading group behavior are 
difficult to Interpret in any case, because they relate teacher behavior In read- 
ing groups to student learning gains on the two mathematics subtests! The one 
significant correlation involving the non-Title I schools Is negative, and the 
general pattern suggests that it Is unwise for teachers in these schools to stay 
with students who provide correct answers to reading group questions and better 
If they move along to another student following a correct answer. 

Th*^ data on persistence In seeking an answer following failure to respond 
correctly are also mixed, although they do allow one generalization: If the teacher 
elects to stay with a student In an attempt to elicit a response. It Is generally 
better if she provides a clue, rephrases the question, or asks a different ques- 
tion than if she simply repeats the original question* This seems to make intui- 
tive sense, since there seems to be little point in simply repeating a question 
unless the student did not hear it the first time or the teacher was sim^ty try- 
ing to pump him to respond. Where the student has clearly heard the question and 
has responded incorrectly or does not know the answer, however, simply repeating 
the question amounts to pointless pumping. In these situations the teacher should 
provide some kind of help, either by making the question easier or by providing a 
clue, and not simply repeating the original question without giving any help. 

Cluster I contains only one variable, but a very important one: the relative 
frequency of public response opportunities. Of the variables in the Table, this 
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one corresponds most closely to the "student talk" variable stressed by Flanders 
and others. Mote that the pattern of correlation supports the contenHons of 
Flanders and others that student talk is Important, but only for the non-Title I 
schools. The one significant correlation Involving this variable for Title I 
schools was negative . The data for this variable are one Instance of a larger pat- 
tern of findings suggesting that generally indirect teaching, frequent student talk, 
and frequent pupl l-to-pupl I Interaction are facilitatlve in non-TltIc I schools, 
but not In Title I schools (more will be presented below). The children In Title 
I schools appear to profit more from more highly structured and teacher-dominated 
instruction. These students, in the early grades at least, apparently require 
more direct Instruction, explanation, and demonstration from the teacher and are 
less likely to profit from discussion and independent activities than children In 
In non-Title I schools. 

Cluster J deals with student-Initiated questions. The frequent absence of 
correlations In this cluster Indicate that student-Initiated questions were rela- 
tively Infrequent, This should be kept in mind in evaluating the following results. 
First, these questions were coded as either relevant or *r re levant to the topic 
under discussion at the time, with the expectation that relevant student ques- 
tions would correlate positively with student learning gains. Unexpectedly, the 
only significant correlations on the relevant versus Irrelevant aspect of student- 
Initiated questions showed that higher frequencies of irrelevant questions were 
associated with learning gains. However, note that this relationship exists large- 
ly in the Title I schools In reading groups. This fits in with several previous 
findings suggesting that any teacher behavior which involves acceptance. or en- 
couragement of voluntary contribution to discussion In Title I schools (especially 
in reading groups) will be oosltlvely correlated with learning gains. This same 
point is relevant to the following measure concerning the percentage of student- 
initiated questions which were called out. Here there was only one significant 
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relationship, but It was unexpectedly positive, and ft was for the Title I 
schools only* 

The data for praise and criticism of student-Initiated questioriS showed 
a mixed pattern, just as the previous praise and criticism data did. Praise of 
relevant student-Initiated questions correlated positively, as expected, but so 
did criticism. These data are for the non-Title I schools only, because praise 
and criticism of relevant student-Initiated questions did not occur often enough 
In Title I schools to allow computation of correlations. Perhaps the praise was 
for questions that 5he saw as having already been answered or as Indicating care- 
lessness or Inattention; we cannot tell from the data. The praise and criticism 
directed towards student call out behavior (as opposed to praise and criticism 
of the questions themselves) were also surprising. As expected, criticisms and 
warnings dir^-^rted at the students for calling out questions without prior author- 
ization were positively associated with student learning gains, but only for rele- 
vant questions. Criticism for calling out Irrelevant questions did not occur fre- 
quently enough to allow computation of correlations, and warnings for calling out 
Irrelevant questions were negatively correlated with student learning gains. Fur- 
thermore, the latter data are for non-Title I schools, so thay do not fit In with 
our earlier statements about the Importance of encouraging any kind of contribu- 
tion to the discussion in Title I schools. The latter correlations are based on 
a very small n-jfnter of teachers, however, so they may well be spurious. We pre- 
fer to wait and see If they replicate In next yearns data before attempting to 
Interpret them. 

The data on teacher response to relevant student-Initiated questions suggest 
that In non-Title I schools the teacher who legitimized the question and then 
either delayed feedback until a more appropriate time or gave brief feedback on 
the spot were more effective than teachers who either refused to accept the ques- 
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tlon or who responded with long feedbeck (probably breaking up tiie pace of the 
lesson). Redirecting the question to the class was uncor related with teacher ef- 
fectiveness, somewhat In contradiction with the advice of proponents of frequent 
student talk and integration of student questions into the discussion. Cor re I a* 
tlons Involving these variables In the Title I schools were generally similar, 
except that there were some negative cor re I ^*t ions for delaying responding to the 
question, suggesting that students In Title I schools need more Immediate feed- 
back than students In non-Title I schools. Also, redirection of the question to 
the class was negatively correlated with learning gains in the Title i schools* 
This Is another Indication of the point made above that the Tiria I students seem 
to need more teacher-dominated Instruction and less pupl l-to-pupi I Interaction 
than the students in non-Title I schools. 

The data regarding teacher feedback +o Irrelevant student -initiated ques- 
tions are extremely mixed and based upon a small number of teachers, so that no 
attempt to interpret them will be made. 

Cluster K also contains a single variable: the percentage of public response 
opportunities which are initiated by students rather than teachers* Contrary to 
expectations, the only significant correlations Involving this varlatia were In 
the non-Title I schools, and these were negative . If taken at face value, these 
data would seem to go against the general finding that Indirect teaching ^n(j stu- 
dent talk appear to be facllltative In non-Titie I schools. However, It Is possi- 
ble that this is a ••proxy" variable for classroom control. Perhaps the teachers 
In non-Title I schools who have very high rates of student- -nl tla+eu comnents and 
questions are teachers who have poor classroom control and are generally disor- 
ganized. This is a very likely possibility, since data from other aspects of the 
study (both In this paper and In Evertson and Brophy, 1973) suggest that classroom 
control, especially the management skills described by Kounin (1970), are amona 
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the most Important correlates of student learning gains in both Title I and 
non-Title I schools. 

Cluster L deals with student- Initiated comments. These were considerably 
more frequent than student-initiated questions, although still relatively in- 
frequent. As was the case with student-initiated questions, the percentage of 
relevant student-initiated comments was negatively associated with student learn- 
ing gains in the non-Title I schools (although only one correlation reached 
statistical significance) However, in the Title I schools the relationship 
was reversed, with the percentage of relevant student-initiated comments corre- 
lating positively with student learning gains. 

The percentage of student-initiated comments which were called out without 
prior recognition of the student was positively correlated vlth learning gains in 
non-Title I schools but negatively correlated in Title I schools. The latter 
correlations constitute an exception to our generalization that teacher behavior 
which encourages student participation of any kind is important in Title I schools. 
Perhaps the called out comments in Title I schools were undesirable (disparaging 
remarks about a classmate's response to a question, for example), or perhaps high 
frequencies of called out comments in Title I schools were associated with poor 
classroom control. Either or both of these factors could have been operating to 
produce the negative correlations. 

Again, the praise and criticism data show a mixed and confused pattern. Praise 
following a relevant student comment was negatively associated with student gains 
in non-Title I schools in one instance and positively associated with students 
gains in Title I schools in one instance. All other relationships were nonsig- 
nificant. Criticisms and warnings fjr calling out comments were unrelated to 
student learning gains in Title I schools, and showed a mixed pattern of corre- 
lations in the non-Title I schools. Praise following an Irrelevant student comment 
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actually correlated positively with student learning gains in one instance in 
the Title I schools, again underscoring the importance of encouraging participa- 
tion and discussions in these schools. Criticism of students for calling out 
Irrelevant comments was unrelated to learning gains in Title I schools but nega- 
tively related in non-Title I schools. The latter findings suggest that the 
more effective teachers in non-Title I schools confine their response to a simple 
warning rather than a more severe criticism when reminding the students not to 
call out comments without permission. 

The data regarding teacher feedback to student-Initiated comments hold to- 
gether rather consistently, although they disagree with the comments of Flanders 
and others concerning acceptance of student ideas. First, failure to give feed- 
back to relevant student comments was not negatively correlated with learning 
gains; in fact, although no relationships were significant, most were positive. 
Furthermore, the various categories of teacher response to relevant student- 
initiated comments showed very few significant relationships to student learning 
gains In non-Title I schools. Delaying a response correlated positively twice; 
not accepting a comment (declaring It irrelevant or out of order) correlated nega- 
tively once; accepting the comment correlated negatively twice; Integrating the 
comment into the discussion never correlated significantly; and shifting the dis- 
cussion to the topic raised by the student's comment correlated negatively once. 
This mixed pattern of findings suggests that giving students feedback (not neces- 
sarily immediately) is important, but not use of student Ideas to the extent of 
adapting the discussion to accommodate them (again, though, bear in mind that 
these data are from the early grades). 

In Title I schools there were fewer significant correlations, partly because 
there were fewer student-Initiated comments* Delaying feedback correlated positive- 
ly once, not accepting the student's comment correlated positively three times, 
and accepting the student's comment correlated positively twice and negatively 
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once. Integrating the student^s comment into the discussion correlated nega- 
tively three times, and shifting the topic to accommodate the student's comment 
correlated negatively once and positively once. Thus the pattern in Title I 
schools again shows the Importance of responding to the student but actually 
shows a generally negative relationship between using the student^s idea to 
the extent of integrating it into the discussion or shifting topics in order 
to accommodate it. The general pattern of findings regarding teacher response to 
relevant student comments suggests that delayed and brief feedback to the stu- 
dent rather than an attempt to use his comment as the basis for discussion seems 
to be the most effective response. 

The data for irrelevant student-initiated comments shows ia similar but more 
extreme pattern than the data for relevant student comments. In non-Title I 
schools, failure to give feedback correlated negatively twice, delaying feedback 
did not correlated significantly, not accepting the student* s comment correlated 
positively three times and negatively twice, accepting or agreeing with the 
student •s comment never correlated signf leant ly, integrating the comment into the 
discussion correlated negatively four times, and shifting the topic to accommodate 
the comment never correlated significantly. Here again, there is no support for 
the importance of accepting student Ideas or accomwodati na them into the discussion. 
There is, however, support for the idea that the teacher should give feedback to 
the student^s comment. In the Title I schools, failure to give feedback and de- 
lay In giving feedback did not correlate significantly, responding to but not ac- 
cepting the student^s comment correlated positively three times, accepting the 
student* s comment did not correlate signf leant ly. Integrating student comments Into 
the discussion correlated negatively four times, and shifting topics to accommodate 
the comment did not correlate significantly. Here again, the major positive cor- 
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relate was non-acceptance of the student's comment; teacher behavior involv- 
ing acceptance of the comment or attempts to integrate into the discussion were 
either uncorrelated or negatively correlated with student learning gains. 

The data in Cluster N regard self-and opinion questions. Self-questions 
did not deal directly with the curriculum, although they sometimes were used as 
lead-ins to a topic ("Do you like bananas?. . .Today we're going to learn about where 
bananas come from.") Other self-questions simply dealt with student likes and 
dislikes or other non-academic matters such as personal experiences, "show and 
tell," etc. Only a few correlations involving se I f-questions were significant, 
and these showed a negative relationship between the frequency of self-questions 
and student ^earning gains. This is simply the obverse of the finding reported 
above (Cluster I) showing a positive relationship between student opportunities 
to answer curricui um-re[ated questions and learning gains. To the extent that 
the teacher is involved in activities which involve only sel f -quest ions, she Is 
not teaching the curriculum. 

Opinion questions are related to the curriculum but they gauge the student's 
opinion on a question, and do not have any simple rignt or wrong answer. They are 
usually used in discussion periods, although they sometimes also are used as a way 
to lead Into a lesson. Correlations regarding the frequency of opinion questions 
as opposed to process, product, and choice questions are mostly but not completely 
negative. Most coefficients are negatl/e^ including 5 of the 6 which were sig- 
nificant. Thus in general the frequency of opinion questions was negatively cor- 
related with student learning gains. In part this is for the same reason men- 
tioned above; when the teachers are asking opinion questions, they are not teach- 
ing the curriculum as such. Also, at this age level the opinion questions fre- 
quently are trivial, not the kind of thought-provoking opinion questions that are 
often asked at the higher grades. 
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Data regarding teacher feedback to opinions expressed by students fol- 
lowing these questions parallel the data given previously. Failure to give feed- 
back was infrequent, but when It did correlate it correlated negatively, sig- 
nificantly three times. Praise correlated negatively whenever the correlation 
reacned significance. Teacher disagreement with student opinion showed a mixed 
pattern, but the one significant correlation was positive. Acceptance of student 
opinions showed one positive and one negative significant correlation, and inte- 
gration of student opinions into the discussion topic showed one positive and 
three negative significant correlations. Here again, the data show the importance 
of giving feedback responses to students but do not support the importance of use 
of student ideas or integrating them into the discussion. 

Cluster N deals with private work and procedural contacts initiated by either 
the student or the tsacher. Correlations for the frequency of private student- 
initiated contacts were positive if the contacts involved work, but negative if 
they Involved classroom procedures or personal concerns. Thus teachers who create 
an atmosphere In which the students feel free to come to them for help or dis- 
cussion of their seatwork are more successful than other teachers in obtaining 
student learning gains. In contrast, teachers who have frequent rates of student- 
Initiated contacts for procedural purposes (presumably because the teacher has not 
developed organizational mechanisms to see that these procedures are handled 
"automatically" so that the students don't have to keep coming up to her to ask 
her questions about them) are less successful In obtaining student learning gains. 

The data on teacher praise and criticism In these private contacts mirror 
the data on these variables In public contacts. For student-Initiated contacts 
Involving work, both praise and criticism were negatively correlated with student 
learning gains. The data on praise in teacher-Initiated work contacts were slml- 
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lar, except that one of the 6 significant correlations was positive. Here 
again, teacher praise and criticism came through as relatively unimportant 
variables, and both tended to correlate negatively with student learning gains 
when they did correlate significantly. Again, there was little support for 
the Importance of praising students, although criticising students did come 
through as a negative teacher behavior. 

The data for teacher feedback in student-initiate d wor k contacts show 
positive correlations for brief teacher feedback and mixed correlations for de- 
laying feedback or giving long feedback. Finer analysis shows that delaying feed- 
back tended to be negatively correlated In Title I schools, while giving long 
feedback was positively correlated In these schools. Thus these students appear 
to need more Immediate feedback than students in non-Title I schools, and they 
sometimes need extended explanations rather than brief feedback. In contrast, 
the students In non-Title I schools could tolerate delayed feedback without 
problems for the most part, and In general did better when given brief feed- 
back rather than long (over-dwell I ng?) feedback. 

The data for feedback in tea cher-initi ated work contacts are different: all 
correlations are negative, whether for mere observation, for giving brief feedback, 
or for giving long feedback. These correlations appear to be less related to the 
feedback categories than to the larger category of teacher-ini tiated feedback, 
which itself seems to be negatively related to teacher success in producing stu- 
dent learning gains. That is, the more successful teachers operated by having 
students come to them when they needed help, and did not spend great amounts of 
time having students work silently while they went around the room checking work 
at the student's desks. The latter method appeared to be especially non-product'. ve 
In the non-Title I schools, as might have been expected. Teachers who use this 
method In effect enforce long periods of silence, passivity, and often Inactivity 
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upon their students. This regimentation appears to be harmful as well as un- 
necessary, especially In non-Title I schools. 

The percentage of student-Initiated contacts which involved personal con- 
cerns (as opposed to work) was negatively correlated with student learning gains, 
as expected. However, the measures of teacher response to these personal con- 
cerns showed differential relationships with student learning gains. Teachers who 
granted a large percentage of these student-initiated requests were generally 
more successful than teachers who tended to either delay or to refuse to grant 
them. These measures In combination probably also reflect the more major teacher 
variable of openness and flexibility regarding student needs versus unnecessary 
regimentation and inflexible structure. 

The measure of private work contacts over Itself plus public response oppor- 
tunities was negatively correlated with learning gains. This fits in with earlier 
data showing that the more successful teachers had more discussion or at least 
question and answer sessions in their classrooms and fewer periods in which stu- 
dents wore involved in silent seatwork or other activities which did not involve 
response opportunities. The measure of procedural contacts over Itself plus re- 
sponse opportunities shows an even stronger set of negative relationships for the 
same reason. 

The measure of teacher-Initiated work contacts over itself plus teacher-Ini- 
tiated procedure contacts somewhat surprisingly also shows negative relationships 
with learning gains. This reinforces the statement made earlier that apparently 
teacher-initiated work contacts are themselves negative in some direct way, and 
that teachers should tralh their students to come to them when they need help 
rather than structure the classroom so that much time is spent Inspecting student 
work. (Note: An alternative or additional explanation of these findings is 
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that the teachers who Initiated a large number of work related contacts 
during schools hours are correcting papers In school and taking up school time 
to do this, while teachers who have low rates of initiated work-related con- 
tacts may be teachers who reserve as much school time as possible for other 
matters and who correct papers In their time out of the classroom. This seems 
reasonable but It Is not directly testable from the available data. It will be 
Investigated in next year's replication study, however.). 

The measure of the percentage of teacher-initiated procedural contacts 
which Involved management requests (as opposed to errands or special favors) was 
strongly negatively correlated with student learning gains. Here again, the data 
suggest that the more successful teachers have set up routines which take care of 
daily management needs "automatically" so that they do not have to continually 
request that these duties be carried out "on the spot." 

The measures of teacher frequency of thanking students following favors or 
management requests showed mixed results, with positive correlations In Title I 
schools and negative correlations In non-Title I schools. Along with the praise 
data, these findings reinforce the more general finding that verbalization of 
teacher affect appears to be relatively unimportant, especially at non-Title I 
schools. It was expected that these two measures would be good Indicators of 
general teacher warmth and rapport with students, and that they would show strong 
and consistent correlations with student learning gains, but they did not. We 
will be Investigating them further to see If this is because the vari^ljles are 
truly unimportant or whether relationships might have been depressed by low var- 
iance or masked by curvl linearity or some other factor. 

Cluster 0 deals with teacher praise, warning, and criticism suwned across 
all of the various contacts that they had with their students. The measure of 
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academic praise (praise of good answers or good work) over Itself plus aca- 
demic criticism shows mixed correlations with student gains. There were two 
negative and three positive relationships in Title I schools, and two positive 
and no negative relationships In non-Title I schools^ The relationships for thi 
particular variable seem to be more determined by the criticism factor than by 
the praise factor. Again, teacher praise was neither as clear-cut nor as impor- 
tant as was expected as a correlate of success in producing learning gains. 

The data on behavioral praise and warnings show negative correlations for 
behavioral praise but positive correlations (with one exception) for behavioral 
warnings as opposed to behavioral criticisms. Taken together, the datsa on 
praise and criticism, both for academic work and for classroom conduct, suggest 
the following: I. contra 'icting the conventional advice and the advice of be- 
havior modifiers in particular, praise does not appear to be a very important or 
effective teacher behavior; 2. however, criticism Is a rather clearly negative 
teacher behavior, with teachers who criticize heavily being the least successful 
3. in general, the successful teachers are those who have good management tech- 
niques which minimize classroom problems and who are we 1 1 -organ I zed so that they 
maximize student learning opportunities and especially opportunities to verbally 
participate in classroom activities. The data regarding classroom management 
strongly support the suggestions of Kounin (1970), particularly the point that 
the teacher's activities in organizing the classroom so that disruptions are min 
Imlzed are far more important than her responses to disruptions which do occur. 
The data regarding opportunity to learn confirm several previous studies showing 
this to be an important variable (Rosenshlne and Furst, 1971). 

Cluster P deals with discipline and control errors made by the teacher. 
These categories were adapted from those used by Kounin (1970). Again, the data 
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generally bear out Kounin and provide little support for behavior modifica- 
tion techniques. The percentage of discipline contacts involving one or more 
errors was generally negatively correlated with learning gains, although there 
was an exception in Title I schools. The measure of target errors was uncor re- 
lated with learning gains (target errors were Infrequent), Timing errors were 
negatively correlated with learning gains (essentially In Title I schools), and 
teacher over-reactions were mixed. These data suggest, as does Kounin, that It 
Is better for the teacher to err on the side of nipping a potentially disruptive 
situation In the bud than to err by allowing It to go on too long and to begin 
to spread to other students. Thus It appears to be better to move In too quickly 
or to overreact than to underreact. The measure of non-verbal control contacts 
over total control contacts which was expected to show positive relationships ac- 
tually showed negative relationships with learning gains. This variable refers 
to stopping disruption or forbidden activity through non-verbal means such as 
gestures or quietly touching the offending student, as opposed to more disruptive 
methods such as calling out the student's name or stopping the activity In order 
to deal with the situation. Advice on classroom control usually suggests that 
this is the most preferable method of Intervention If Intervention is necessary, 
since It is the least disruptive. However, correlations with learning gains were 
negative, reinforcing the point made above that the data suggest that it Is better 
for the teacher to act quickly and decisively than to delay or underreact to a 
control problem. 

Cluster Q, the final cluster, shows teacher feedback data combined across 
various types of contacts with students. These data show more clearly that sim- 
ply repeating a question when the student has not been able to respond the first 
time (pointless pumping) Is Ineffective, while rephrasing the question and giving 
a clue are effective. Also, brief feedback Is generally more effective than 



long feedback, although there Is one exception In Title I schools. In qener- 
al, the non-Title I students seemed to require less extended and less Imme- 
diate feedback than the students in Title I schools, and correlations for 
the various teacher feedback categories usually were In opposite directions 
for these two types of schools* The most likely reason for this is that stu- 
dents In non-Title I schools are probably able to work Independently more 
successfully than students In Title I schools, and probably are less In need 
of feedback or guidance from the teacher* Thus brief feedback Is sufficient 
to meet their needs and enable them to return to Independent work. Students 
In Title I schools, however, often require more extended teacher feedback be- 
fore they can profitably return to Independent work* 
Small group instruction variables* (See Appendix B) 

Data from the coding system for small group Instruction variables are pre- 
sented In Table 3, which Includes both correlations between process measures 
and student gain measures as well as correlation coefficients reflecting ob- 
server agreement* Relatively few of the relationships of these process variables 
to student learning gain measures reached statistical significance, but It should 
be borne In mind that data were available on 10 teachers (8 in the case of the 
Arithmetic Reasoning scores), and that teachers were observed only twice, so 
that the process measures were very likely weakened by error variance due to 
low rel labl I ity* 

The teachers selected for observation with this Instrument were five of 
the most effective and five of the least effective in the sample* This selec- 
tion was made because limitations on time and personnel required limitations 
on the number of teachers who could be observed with this system* Observers did 
not know the effectiveness of the teachers, of course* 

Correlations are presented only for the whole group because only three of 
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the 10 teachers Included were teachinq in Title I schools. In effect these 
data for the whole group represent the relationship in the non-Title i schools 
primari ly. 

Data on the lecture versus discussion mode of presentation suggest that 
this dimension is unimportant to student learning gains or else is related to 
them in some more complex way than sheer frequency or degree of emphasis. 
There were significant relationships relating teacher behavior in small group 
lessons to student learning gains, but they did not include the amount of 
time spent lecturing or discussing. One variable that was Important was the 
relative amount of time that the teacher spent lecturing or introducing a topi 
versus allowing the students to practice t' o newly introduced content. Teach- 
ers who spent great amounts of time reviewing and/or introducing new topics 
were less successful than teachers who spent less time doing that and saved 
more timo to allow the students to practice application of the new concept or 
skill themselves. These findings support the frequently made sungestion that 
young children need to learn by doing. 

Variables relating to whether the teacher worked with individuals, sub- 
groups, or the entire group were strongly related to learnlnq aains and highly 
consistent. The most successful teachers worked primarily with individuals 
and least with the entire qrcup. In combination with the above findings, the 
data suggest that the most successful teachers introduced a new tooic, then 
allowed students to -iork with It and gave Individualized feedback as they ob- 
served their work. 

When the teacher was introducing a topic, she was more successful if she 
used a demonstration or diagram than if she confined herself to lecturing. 
This fits with typical teaching advice based on the Idea that young children 
respond better to concrete than abstract learning presentations. 



34 

Several measures of activities during the groups aqain sugqest the Impor- 
tance of active grappling wifh the material on the part of the students. Si- 
lent reading was positively associated with learning pains, while dead spots 
due to interruptions were negatively associated. The one significant correla- 
tion involving drill activities Intended to promote overlearninq was negative, 
but this finding is tenuous because such activities were observed only rarely 
and in only a few groups* 

The data on patterned versus non-patterned turns flatly negated our ex- 
pectations and the usual advice given to teachers to avoid patterned turns. 
Teachers who follow this advice by using non-patterned turns were less success- 
ful than those who used patterned turns. This is another place where we wonder 
whether the relationship is actually between the patterned turn versus non- 
patterned turn variable and student learning gains or whether the patterned 
turn variable might actually be a "proxy" variable. That Is, perhaps the 
teachers who used patterned turns tend to be teachers who also establish good 
classroom management routines and are generally better organized than the 
teachers who do not. This possibility will be investigated. 

Other significant correlations Included a positive relatfonshlp between 
using traditional materials (those that come with the books) for seatwork and 
homework assignments, positive relation! :lps for the use of games and special 
activities to promote interest and variety, and the use of differentiated In- 
dividualized materials for different children. These findings all reflect the 
most common advice in teacher training materials. 

The percentage of wasted lesson time correlated negatively and the percen- 
tage of time in which the teacher was doing two or more things correlated posi- 
tively, supporting the observations of Kounln (1970). 



[ 



1 



I 



35 



Somewhat surprisingly, the measure of the teacher wo'-King alone removed 
fronr. the class correlated positively (once significantly) with student ream- 
ing gains. We had expected a negative relationship, reasoning that such be- 
havior would reflect avoidance of, or poor rapport wl^h, students. However^ 
coder judgments of rapport with students did not correlate significantly with 
student learning gains (Evertson and Brophy, 1973). In -jny case, no interpre- 
tation will be offered for the finding concerning teachers work'nc alone at 
the desk until we have had a chance to Investigate Its r.)eanlng further. 

Variables toward the end of the table mostly reflect In a somewh<!Jt different 
way the rea^tlonshlps already mentioned (Importance of working with Individuals 
rather than groups. Importance of providing variety of materials. Importance of 
allowing maximum time for utt^dents to practice wr n newly letsrned skills or 
materials, the negative effect of wasted time). Thus on!y two more varlaMes 
iroc Table 3 will be discussed. 
I Variable 52, the percentage of student self-evaluation which Is teacher- 

elicited, showed positive correlations with student learning q?<ns. In essence, 
this means tnat teachers who frequently asked questions relevant i-o the material 
I* Introduced were more successtul than teachers who questfoned Infrequently, This 

fits with the earlier reported findings aoout the Ifr.|rortance of public response 
1^ opportunities. Thus although discussion Is not important for student learninn, 

teacher questioning is. 

Variable 61, a high proportion ol, standardized (vs. teacher-created) materials, 
j correlated positively with student learning gains (once significantly). This was 

mildly surprising, and we are not sure yet what It means.^ If tak^^n at face value, 
I It means that the materials provided In a Published currlcglum are likely to be 

better then those n^ade by the teacher, on the average. However, this relation- 
I ship might also mean that teachers who use a relatively high percentoje of teach- 

EjRX^ er-made materials are using lesson time unproducti vely, such as by playing games 
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or conducting drill activities of questionable value to the students* In any 
case^ this variable needs to be investigated further before Its meaninq can 
be interpreted with confidence* 

DISCUSSION 

The above list of behaviors correlating positively with teacher success 
In producing student learning gains contains many that are frequently advocated 
In teacher Training materials. However, It also contains a few relationships 
which contradict the typical advice given to teachers. This is especially true 
for the data from Title I schools. The importance of school SES appeared In a 
two-way analysis of variance (Title I vs. non-Title I classrooms and grade 2 
vs. grade 3 classrooms as the classifying variables) carried out on all of the 
measures available at the time. These analyses showed that although grade was 
relatively unimportant (the main effect was significant for only 4Jt of the 
variaoles), the main effect for Title I vs. non-Title I classrooms was significant 
for 25% of the measures. Furthermore, a grade X Title I Interaction appeared on 
\0% of the measures, and most of these appeared to be caused primarily by Title I 
rather than grade differences. 

These analyses suggest that what is optimal teaching in Title i schools 
Is not quite the same as what Is ootimal teaching in non-Title I schools, and 
point up the need for separate anal/se> of process-product data taken in schools 
of contrasting SES. In general, the -isults for non-Title I (high SES) schools 
bear out the typical teacher trair ig text's advice, but those for Title I schools 
often do not. 

Second year replication 

Considering the carefully selected sample of teachers on which they were 
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based ^ and the broad variety of both hlqh and low inference measures included 
In the analyses, the data described above (along with those reported in Evertson 
and Brophy, 1973) represent a qualitative improvement in several respects over 
previous teacher effectiveness research. The obvious next step in the research 
and development process is to move frOiT^ the naturalistic model which identifies 
behavioral correlates of effective teaching into experimental models designed to 
establish the cause and effect relationships among the variables involved. How- 
ever, this step has been deferred for a year in order to get a second year of 
natural i stic data. 

This repetition of the naturalistic data collection cycle was deemed 
necessary for several important reasons. First, the process measures were based 
on a sample of only four observations per teacher (and in the case of the group 
presentation methods observations, on a sample of only two observations per 
teacher), and these observation frequencies are dangerously small. Given the 
probability that teacher behavior varies from one observation to the next, because 
of the situational factors operating (whether the teacher happens to be intro- 
ducing a new unit or reviewing one before finishing it, for example), the im~ 
pression gained by observing a teacher only four times m*^ oe somewhat in- 
accurate, even if she is basically consistent in her classroom behavior. Con- 
sequently the behavioral coding and high^-inference ratings will be used again 
during a second year of naturalistic observation. 

The second year of naturalistic data collection will include ten to fifteen 
half day observations per teacher Instead of only four. This will be accomplished 
partly by starting earlier, and partly by using only a single observer in the 
classroom rather than pairs, once reliability is established. Each teacher will 
still be seen by at least two observers (separately) even in the second year data 
collection, however, so that interobserver agreement on the high-inference ratings 
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can be checked once again. 

Another reason for repeating the naturalistic phase before going on to 
experimental paragigms is that the first year's data set is completely unique 
and In many respects not comparable to any other data. Also, since only 31 
teachers were studied and the research involved well over 1000 variables on each 
teacher, the study obviously violates several assumptions underlying the use 
of significance tests, so that the significance tests used do not provide the 
logical basis for drawing the inferences that they would provide when the 
assumptions underlying them are met. 

The replication study, of course, will also violate the same assumptions, 
since it will also involve a relatively small number of subjects and a larqe 
number of variables; however, it will provide a second set of data that can be 
compared directly with the first, and ultimately, replication of a set of find- 
ings across several studies rather than the level of significance of a finding 
In a single study is the basis for acceptance of empirical data. Those findings 
that replicate across both years of naturalistic data collection are likely to 
be quite solid and to replicate in future studies also, despite the problems in 
applying inferential statistics to either data set. 

Most of the teachers studied in the first year will be studied again in 
the second year (a few have retired, gone on maternity leave, or been transferred 
to other grades), so that data bearing on the question of stability of teaching 
style across two years will be generated. In addition, new teachers will be 
added to the study according to the same criterie used for selecting those in 
the original sample, so that the replication will include some teachers wno were 
not studied the previous year. Inclusion of these new teachers will help us 
determine whether the process-product relationships shown in the first year data 
are general izable and not restricted to the particular sample of teachers studied 
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the f i rst year. 

In addition to using the same high and low inference measures of class- 
room process used the first year, the replication study will add some new 
measures to help round out the picture of effective teaching that we are gather- 
ing. Some of the new data will come from additional hiqh- inference ratings 
of variables not rated In the first year. Most of the new data, however, will 
come in a teacher interview to be administered at the end of the second year. 
This interview will cover a broad range of content, probing the teachers on their 
philosophies, approaches to teaching, time utilization in and out of class, 
planning and organization, curriculum and methods preferences, and many other 
matters. Particular attention will be given to those aspects of teaching that 
cannot be picked up through only periodic observation, especially events that 
unfold over time, such as the planning and teaching of entire units and the 
teachers* methods of dealing with particular kinds of children (the bright child 
who finishes his work early and needs some extra assignments, the slow child who 
needs extra time from the teacher, the child who misses a week or two because of 
illness and has to catch up, etc.). 

Other Analyses 

In addition to replication, the present data require further analyses 
designed to reveal their full meaning. First, the present data are restricted to 
Pearson £'s, which reflect only linear process-product relationships. There is 
reason to believe that several relationships will be curvilinear or otherwise 
non-linear. This will be investigated in a series of multilinear regression 
analyses of the process-product relationships. 

The variance In each process variable also needs to De analyzed, especially 
for variables which "should have" correlated significantly. It may be that lack 
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of correlation was because of limited variance In certain process measures, rather 
than to a genuine absence of a process-product relationship. 

I ntercorrelatlons and possibly factor analyses of the measures are also 
needed, to help reveal their interrelationships. Even though there were many more 
variables than subjects, such analyses will be useful, especially for revealing the 
precise meaning of certain relationships (such as separating those which appear to 
be genuine process-product correlations from those in which the process measure 
appears to be a "proxy variable," so that the actual relationship is somewhat differ- 
ent from that suggested by the correlation taken at face value)* 

Caveats 

The data are mostly internally consistent regarding process- pro duct relation- 
ships, although a few apparent contradictions appeared. Also, the findings reported 
in this paper are regularly supported by the high-Inference viirlables In the Evertson 
and Brophy (1973) paper. The only clear exception Is the variable of staying-at- 
desk vs. golng-around-the-room while correcting seatwork. The present data suggest 
that staying at the desk and having children come to the teacher is preferable, but 
some high-inference data suggest the opposite. Thus, in general the process-product 
data hang together rather well, even though they often contradict the conventional 
wisdom about effective teaching (Including many of our own expectations! ) • Never- 
theless, all of the following factors, each of which represents a potentially Impor- 
tant source of error , should be borne in mind in Interpreting the results: 

I. Only two-four observations per teacher were obtained on a sample of only 

31 teachers (really two subsamples of 18 and 13, since Title I and non-Title I 

schools appear to require separate analyses). 

a. Thus the low-Inference process measures undoubtedly contain much error 
variance due to situation variability. This problem will be addressed In 
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the replication study by Increasing the number of observations to 
10 to 15. 

b. The high-inference ratings, checklists, and percent estimates (Evert- 
son and Brophy, 1973) show signs of widespread halo effects and/or logical 
errors, despite coder training. This also probably stems from the limited 
contact each coder had with each teacher, and will be reduced In the 
repi ication study. 

2. Most of the statistically significant £'s are low, the significance level 
was set at .10 instead of .05, and assumptions underlying the use of signifi- 
cance tests could not be met. Thus many £'s are probably spurious and will not 
rep I Icate. 

3. Without replication, and in the absence of comparable data, it Is diffi- 
cult to predict which £'s are meaningful at face value, which are genuine but 
cannot be taken at face value because they involve "proxy variables," and which 
are spurio*:s and will not replicate. 

4. The product criteria probably were inappropriate to some unknown degree for 
each teacher, to the extent that her curriculum objectives differed from or 
were not tested by the MAT. Thus it is possible that some teachers with low 
product gain scores were actually achieving good product gains, but in areas not 
measured by the MAT. (This is not likely to be a very serious factor, however. 
Gains In various areas correlate strongly in the early grades and teaching 
stresses the three R's, so that MAT gains are probably good estimates of teacher 
effectiveness. Also, the pattern of findings, especially the poor management 
skills and low opportunity to learn observed in classes of teachers with low 
gain scores suggest that the criterion scores are accurate). 

5. Only process-product data have been presented; so far we have not Investi- 
gated the full meaning of the product (learning gains) criterion. Other data 
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(Peck and Veldman, 1973) suggest that achievement gains might be achieved at 
a cost In affect or other areas, judging by the personal characteristics and 
attitudes of the teachers* This question wi II be addressed in the correlational 
analyses relating process variables to one another as well as to product cri- 
teria. In addition we wi 1 1 be administering a measure of student attitudes to- 
ward the teacher. In particular, we will attempt to see if the teachers who 
get consistently high learning gains are "slave drivers," "charismatic teachers," 
*moti vators," or what. At present, we don't know* 

Implications for teacher evaluation 

The project is not far enough aloncj yet to allow us to draw many clear-cut 
implications for teacher evaluation, but a few statements can be made with confidence 
at present. First, although the Brophy (1972) teacher stability data were encourag- 
ingly higher than those previously reported, and although they allowed Identification 
of consistent teachers to be included in the observational studies described, the 
stability coefficients for the sample as a whole were not high enough to justify the 
use of standardized achievement tests for evaluating teachers . Thus, In an unselected 
sample there is simply too much variability from one year to the next in teachers' 
production of student learning gains on these tests. 

Second, the Brophy (1972) data show that a yearly "class" or "cohort" effect 
is noticeable even when residual gain scores which are supposed to eliminate such 
effects are used. Residual gain scores from different subtests In the same year 
Intercorrelate more highly than scores from the same subtest correlate from one year 
to the next. Such yearly variability might be due to rather obvious causes such as 
teacher and student health or personal problems or changes in curricula, but they may 
also be due to factors which are more difficult to Identify, such as year-to-year 
differences In class leadership, class morale and motivation, or the general tenor of 
teacher-student relationships. 
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Third, for the reasons mentioned above, the findings of the study remain too 
tentative at this point to provide a solid basis for evaluatlnq teachers throuqh 
process observations. However, after two years of data have been collected and all 
of the planned statistical analyses have been completed, we should be able to iden- 
tify teacher behavior that Is related to the production of student learning gains with 
much greater confidence. The results will remain correlational, al+hough those that 
replicate over two years and show up consistently on several different types of 
measures are very likely to be causes and not merely correlates of student gain. 
This will be Investigated, however. Following the second year of naturalistic study, 
we will move Into a series of experimental and quasi -experimental studies in which 
teacher behavior that appears to be related to student learning will be systematic- 
ally varied to see if predicted effects on students are observed. In these studies 
the teachers will be the experimenters and we will be the data collectors, .'sing 
observation Instruments to link teacher and student behavior and demonstrate :ausal 
mechanisms where they exist. These data, and those produced by other investigators 
In similar studies, should eventuate in the development of observation scales that 
can be used to evaluate teacher behavior (for examples, see Good and Brophy, 1973). 

Before such process evaluation can be done with confidence, however, and cer- 
tainly before it should be used for teacher accountability purposes, two advances 
beyond the present state of the art must be made: (I) We must identify teacher pro- 
cess variables which show stable and reasonably high correlations with criteria (what- 
ever criteria are used); (2) Teachers should show high stability on the piocess 
variables themselves, in the absence of intervention or treatment designed to change 
their behavior on these variables. Where these conditions were met, teacher evalua- 
tion through process observation would be quite valid and defensible. Even here, 
however, process observation instruments should not be used for evaluation purposes 
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only. Training modules desiqned to optimize teacher behavior on each variable 
should be developed and used with the teachers, and where accountability is involved, 
teachers should be rewarded for making gains on these variables of maintaining a 
high level of performance. It would be tragic and wasteful if process evaluation 
data that had such obvious implications for inservice teacher training were used 
solely for accountability evaluations. 



References 



Brophy, J. Stability in Teacher Effectiveness. Report Series No. 77, Research 
and Development Center for Teacher Education, The University of Texas at 
Austin, 1972. 

Brophy, J. and Good, T. The Brophy-Good Dyadic Interaction System. In A. Simon 
and E. Boyer (Eds.). Mirrors for Behavio r ; An Anthology of Observation 
Instruments Continued, 1970 Supplement, Volume A, Philadelphia: Research 
for Better Schools, Inc., 1970. 

Coleman, J., ©t aj^. Equality of educatio n al opportunity . Washington; U.S. 
Government Printing Office, 1966. 

Dunkin, M. and BIddle, B. Book on Teacher Effectiveness. In press, 1973. 

Evertson, C. and Brophy, J. High inference behavioral ratings as correlates of 

teaching effectiveness. Paper presented to the annual meeting of the Ameri- 
can Educational Research Association, 1973. 

Flanders, N. and Simon A. Teacher Effectiveness. In R. Ebel (Ed.). Encyclo- 
pedia of Educational Research , 4th edition. New York; Macmillan, 1969. 

Good, T. and Brophy, J. Looking In Classrooms . New York: Harper and Row, 1973. 

Hess, R. Class and Ethnic Influences upon Socialization. In P. Mussen (Ed.). Car- 
michaeTs Manual of Child Psychology , Volume 2. New York; John Wiley and 
Sons, 1970. 

jencks, C, et aj[. Inequality; A Reassessment of the Effect Family and School- 

Ing In America. New York; Basic Books, 1972. 
Kounin, J. Discipline and Group Management in Classrooms . New York; Holt, Rine- 

hart and Winston, 1970. 



Mitzel, H. and Gross, The dev^iopment of pupil growth criteria in studies of 
teacher effectiveness. Educational Research DuMetin , 1958, ITj 178-87; 
205-275 • 

Mood, A, Do teachers make a difference? In A, Mood (Ed,). Do Teachers Make a 
Difference? Washington: U.S. Government Printing Office, 1970, 

Morsh, J. and Wilder, B. Identifying the Effective Instructor; A Review of the 
Quantitative Studies, 1900-1952 . USAF Personnel Training Research Center 
Bulletin No. AFPTRC - TR - 54 - 44, Lackland Air Force Base, San Antonio, 
Texas, 1954* 

Peck, R, and Veldman, D. Personal characteristics associated with effective 

teaching. Paper presented to the annual meeting of the American Educational 

Research Association, 1973. 
Rosenshine, B. Teaching Behaviours and Student Achievement , London: National 

houndation for Educational Research in England and Wales, 1971. 
Rosenshine, B. and Furst, N. The use of direct observation to study teaching. 

In R. Travers (Ed.). Second Handbook of Research on Teaching. New York: 

Rand McNal ly, 1973. 

Veldman, D. and Brophy, J. Measuring teacher effects on pupil achievement. Paper 
presented at the annual r.ieeting of the American Educational Research Associ- 
ation, 1973. 



r 



Table 1. Intercoder Agreement on the Expanded 
Brophy-Good Observation System 
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A. Aqrooment Percentages 
Behavior Category Requiring Coding Uoci.ion (presence 
absence and/or choice of which alternative applies) 

Public Response Oppor tunities 

(names respondent before question vs. calls on non- 
volunteer vs. calls on volunteer vs. student calls out 
answer) 

Level (process vs. product vs. choice) 
Answer (correct vs. part-correct vs. incorrect vs. 
"don't know" vs. no response) 

Feedback (praise vs. criticism vs. no feedback vs. process 
feedback vs. product feedback vs. calls on another child 
vs. student calls out answer vs. repeats question vs. 
gives clue vs. asks new question) 

«^tudent-lnitiated Questions 

Relevancy (relevant vs. irrelevant) 

Feedback (praise vs. criticism vs. no feedback vs. delays 
feedback vs. does not accept question vs. brief feedback 
vs. long feedback vs. redirects question to class) 

<;tiident-lnitiated Comments 

Re levancy (relevant vs. irrelevant) 

Feedback (praise vs. criticism vs. no feedback vs. delays 
feedback vs. disagrees with comment vs. agrees with comment 
vs. integrates comment into discussion vs. shifts topic to 
topic of student's comment) 



% Agreement 
f^dian Range 



63 37-78 
73 46-93 

70 52-86 



58 1 7-69 



52 



43 



53 



43 



29-71 



29-73 



38-83 



25-78 



I 



B. Correlation Coefficients Median Range 

j Student^lni tiated Questions 

Type (sanctioned vs. called out) .SI •04-. 90 

I 

Student- In i tiated Comments 

I Type (sanctioned vs. called out) .94 .87-. 99 

1 Child-Initiated Work Contacts 

I 

Feedback 

j Praise .80 .41-. 95 

J. Criticism .86 .09-. 99 

^ Delays Feedback .64 J2-.89 

I* Brief Feedback .97 .88-1.00 

Long Feedback .90 .81 -.99 

Teacher- Initiated Work Contacts 



I 

I 
1 



i 
1 
1 
I 

ERIC 



Feedback 

Praise .90 .79-. 99 

Criticism .66 .22-. 95 

Merely Observes .96 .59-1.00 

Gives Brief Feedback .94 .18-. 94 

Gives Long Feedback .95 .75-. 99 

Child-Initiated Procedure Contacts 
Type 

Personal .84 .33-. 99 

Managerai .83 .28-. 98 
Response 

Grants request .58 .22«.95 

Delays request .50 .11-. 87 



^"ed I an Ranoe 

Does not grant request .83 .12-1. CO 

Teacher^lnitlated ProceJure Contacts 
Type 

Favor request .86 .55-1.00 

Management request ^91 .33-. 98 

Response (thanks child vs. does not thank child) .69 .19-1.00 

Behavior-Pi scipH iO Contacts 
Teacher Behavior 

Praise of good behavior .94 .53-. 98 

Non-verbal warning re^ mi sbehavior .64 .05-1.00 

Verbal warning re mIsbeKavJor .89 .68-. 99 

Criticise * misbehavior .78 -.16-. 99 
Errors in Hanc'iinq Control Probiem5 

No Error .72 .15-. 93 

Target error .89 -.08-1.00 

Timing error .56 .34-. 93 

Over**eaction .80 .23-. 9^ 



T 



Coders worked in pairs. Where coding sequences could be matched and compared 
(Section A), exact agreement percentages are given. The remaining variables 'Section 
B) were simply tallied, so that agreement percentages could not be computed ^r.i 
Pearson £'s between the totals for each classrootjt visit ware computed instead. 
*^tgures c^re for pairs of coders; thus the range data show the differences between 
the least and most reliable pair of .^ders for each item. 
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Table 3. Correlations between Group Instruction 
Coding System Variables^ and Residual Cain Scores, 

Word Word Dis- Arithmetic Arithme- Rater^ 
crlmlna- Read-Computa- tic Rea- Agree- 



I&iow- 
ledge 



tlon ing tion sonlng ment 



Group Instruction Variables -(N=10) (N=10) (N=10 ) (N=10) (N=8) 



1, Reviewing Old Material 



-50 



-41 



during Lessons 

2, Presenting N^w Material 
during Lessons 

3, Practicing New Material Just 
Presented during Lessons 53 

k, Beviewirig or Summarizing 

Lessons -23 
5. Evaluating Work or Responses 
during Lessons 38 
j 6.Eliciting Self-Evaluation 



from Students d uring 
Lessons 

7. Giving Assignment Instruc- 
tions during Lessons 

8. Allowing Students to Work 
Independently during 
Lessons 

9. Working with Individuals 
during Introa.uctions to 
Lessons 



^"•0, Working with More than 
nlta One Child but not the Whole 



00 



05 



39 



58^ 



-28 



55* 



'05 



11 



08 



05 



16 



-36 -43 



-48 -48 



41 



42 



-13 -03 



53 39 



-04 19 



30 



-c 



00 



22 



13 



70«* 4? 



-59 



-68» 



37 



-04 



46 



36 



-03 



59 



59 



67 



78 



•51 



-19 



87 



04 



61 



to Lessons 


-3^ 


-07 


-11 


Oo 


-10 




j 11, Working with the Whole Group 












during Introductions to 














1 

! Lessons 




-15 


-69** 


If O 


-58 




! 12. Presenting Demonstrations 


or 










30 


1 

Diagrams 


ko 


35 


^3 


59* 


52 




47 


1 13. Lecturing 


-31 


-17 


1 A 
—1 0 




02 


Ik, Focused Discussion 


30 


27 




33 


O Jt 

34 




i 15. Unfocused Discussion 


-39 


-l^f 


-19 


03 


-07 




16. Pupils Read, Recite, or 














Work Problems 


06 


03 


-11 


-17 


-19 


AO 
92 

• 


17. Pupils Read Silently 


56* 


35 










18. Drill Activities to 












1 


I Promote Overleaming 












00 


— 












j 19. Thinking and Problem 














Solving Activities 


12 


-27 


-07 


^ la 

-14 


/\jt 


ox 


j 20. Dead Spots Due to Inter- 














j ruptions 


-58* 


-51 


-71** 




■•ox 


^ 21. Patterned Turns 


39 


30 


22 


19 


03 




j 22. Non-Patterned Turns 


-^7 


-15 


unit 

-57* 




— f w 


93 


23. Working with Individuals 














i during Lessons 


ifS 


06 


^7 


11 


36 




2^. Working with More Than One 










• 


' Child but not the Whole Group 












j during Lessons 


-23 


-07 


-0*f 




—W f 




25. Working with the Whole Group 












1 during Lessons 


-^7 


-05 




-13 


-37 




j 26. Use of Standardized, Pub. 


m 












' lished Materials during 














:RJC Lessons 


.19 


-06 


-1^ 


-23 


-^3 


90 



27, Use of Teacher Created 

Materials during Lessons ^6 38 28 17 13 87 

23. Use of Traditional Mater- 
ials for Seatwork or Home- 

work 57* 08 38 19 ^2 81 

j 29 Use of Audiovisual Aids 

' during Lessons 38 53 26 30 ^5 1.00 

I 

I 30, Games and Special Acti- 
vities ^ 51 59* 58* 79** 52 
' 31, Use of Learning Centers -25 -35 -1^ 05 17 89 
I 32, Allows Students Free Choice 

of How to Spend Free Time -19 -^3 -27 -25 — 1«00 

I 33, Prepares Individualized 
I Materials for Different Chil- 

I dren for Lessons 56* 20 65** 32 ^6 

1 34, Uses Different Materials 
for Different Groups for 
[ Lessons -21 '^3 -30 -12 05 

35, Uses Same Materials for 

All for Lessons 18 35 10 00 -13 

I 36, ^ Total Time Spent with 

Individuals 55* 17 . 65* 30 ^5 96 

1^37* % Total Time Spent w ith 

I Small Groups -22 -^5 -30 -I6 02 97 

" 38. % Total Time Spent with 

I Entire Group -39 U -^3 -19 -^8 90 

39, Wasted Lesson Time -6I* -^3 -55* -32 -53 

1 ^0, ^ Total Time Teacher Doing 
Er|c Thing at a Time -37 -H -36 -26 -29 

/i-i < "Pc^trt"* Tipe Teacher Doing Two 



5 

j or More Things Simul- 
taneously 6^** 22 62*» 40 50 

I kZ, Teacher Works Alone, Re- 
moved from Class 31 11 17 n 66* 

^ 43. Teacher Uses a Single 

j Approach to Present a 

Lesson -24 05 -I3 15 00 

I 44. Teacher Uses Multiple 

I Approaches to Present a 

' Lesson 13 -10 06 -21 -16 

I 45. Teacher Monologue (Lec- 
ture, Chatter, Admonish- 
I ment) -39 -08 -26 -03 -17 

I 46. ^ Total Time Teacher Uses 

Same Materials for All 
j Students -I3 -09 -19 -26 -42 

47. % Total Time Provides Vari- 
i ety or Student Choice of 

I Materials, Activities 62** 20 51 30 64* 

•• 43. % Instructional Time Used 

j for Review -29 09 -09 -22 -37 

49. % Instructional Time Used 
I Presenting New Material -01 -44 -10 -25 -11 

j 50. ^ Instructional Time Used 

for Student Practice of Newly 
] Presented Material 59* 64** 47 46 44 

51 » % Instructional Time Used 
1 Summarizing Presentations -26 -08 -I6 00 -05 

ERIC 



^ n % of student Self Evaluation 



Which is Teacher Elicited 30 10 3^ 6I* 79** 

53, % of Lecturing Which In- 
cluded Demonstrations 43 30 29 29 18 

Direct Presentation of 
Material/Total Modes of 

Teaching -04 -06 02 28 29 

55, % of Discussion Which Is 

Focused by Teacher ^5 0? I6 -11 0? 

56, Discussion/Total Modes of 

Teaching -17 04 -02 05 -13 

57, Pupil Recitation/Total Modes 

of Teaching -13 -1^ -35 -41 -43 

58, Oral Reading/Oral Plus 

Silent Reading -10 -08 -25 -10 -24 

59, Wasted Teaching Time/Avail- 
able Teaching Time -32 -15 -57* -40 -57 

60, Patterned Turns/Total Turns 58* 12 52 28 3$ 

61, Standardized/Standardized 
Plus Teacher Created Mater- 
ials 61* 46 40 29 33 

62, ^ of Instruction Given Via 

Demonstrations 03 25 14 13 29 

63, % of Instruction Given Via 

Lecturing 31 03 35 21 43 

64, ^ of Instruction Given Via 

Discussion 10 02 -15 -18 -56 

65, % of Instruction Given Via 

Pupil Turns ) 14 10 04 -11 -19 

66, % ot Independent Activities 

Preceded by Teacher Instruo 02 06 -11 -22 -33 

tions 
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Low inference variables coded during observations when teachers were 
I primarily involved in group teaching. Two observers were used to estab- 
j lish reliability, but scores are from just one observer (an experienced 
teacher), 

I ^Reliability data are Pearson r's for the basic categories. Variables 
without reliability data were constructed arithmetically from the other 

I 

i variables, 
I * P<.10 
** p<,05 
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GBNERAL PROCEDURE AND ORGANIZATION OP CODING 



!• Pill In all the Information at the top of each and every sheet* 

2. Use a separate sheet for Rcfsponse Opportunities for each subject. 
Do not draw lines • use (and label) a separate sheet. 

3. Use a separate sheet for Response Opportunities and a separate sheet 
for Child Created Contacts (CCC) and Teacher Afforded Contacts (TAC) 
for reading groups. Label the sheet ^'Reading Group.'* On the CCC an 
TAC sheet draw a line across the entire page near the bottom to 
separate Inslde-group contacts from outslde-group contacts. Outside- 
group contacts go below the line at the bottom of the page. 

4. Transitions are to go on the same sheet as the preceding activity, 
but separated from the activity by a line across the entire page. 
Transitions must be labeled transition and one of these categories 
of transition designated: (1) entire class, (2) Interchange between 
classes, (3) Intraclass group changes (i.e. reading group or math 
group). 

5. Keep an accurate record of the cime. Record time at the beginning 
of each new activity. 

(. When you fill up one section on a sheet, although the other sections 
may be blank, begin a new sheet for all sections. 



ft 



The essential thing to remember when coding is that you must divide 
and label your coding so that it will be meaningful and useful later. 
We must be able later to match your coding by time and activity with 
chat of your partner in order to get the most accurate picture of what 
went on in the classroom and in order to establish inter-coder reliability. 



ACADEMIC 
RESPONSE OPPORTUNITIES 



The coding of response opportunities is perhaps the most difficult 
coding in the system, since several aspects of the interaction have to 
be coded and the sequence of events within the interaction must be main- 
tained and indicated in the coding. To some extent the sequential as- 
pects have already been designed into the coding sheet, since in going 
from left to right the coder takes up coding decisions in the order in 
which they tend to occur naturally: first, he places a "1" for a male 
and a "2" for a female ip the column indicating the kind of question the child is 
responding to; then he codes the level of question; then he codes the 
quality of the child's answer; then he codes the teacher's feedback to 
the child's answer. Each of these aspects of coding response opportunities 
is described in turn below, after clarification concerning the term 
"response opportunity." 

Three key aspects characterize "response opportunities" as they 
are defined in this system; (a) they are public interactions between 
the teacher and only a single child at a time, but nevertheless meant 

4 

for and monitored by the entire class or by the entire group operating 
at the moment (such as the reading group); (b) they occur when the 
teacher asks a question demanding a verbal response from the child or 
when she asks the child to publicly respond to a question requiring a 
non-verbal response (such as indicating something on the board, 
pointing to the right letter or word, etc.); (c) only a single individual 
child makes the response (chorus or unison responses in which two or 
more children call out the answer simultaneously are not considered 
"response opportunities"). Thus a response opportunity involves a 
public attempt by an individual child to v*jal with a question posed 
by the teacher ♦ 

Other types of teacher-child inteiaction are not coded as ''response 
opportunities" because they differ from the preceding definition in 
one or more ways. It is important for coding validity to bear in 
mind that "response opportunities" as used in this system are considered 
to be teacher afforded; it is assumed that the teacher explicitly or 
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at least implicitly wants the child involved in the interaction to 
answer the question. Response opportunities are deliberate teacher 
attempts to get a child to respond, or at least implicit; teacher 
encouragement in situations where the child seeks out a response 
opportunity (see "call out" below). Response opportunities thus 
involve individual recognition of the child by the teacher. The 
previously mentioned situation in which two or more children call out 
an answer simultaneously is not considered a "response opportunity" 
because no individual child receives individual recognition or feed- 
back. Even if only a single child calls out the answer, a response 
opportunity is coded only if the teacher responds to him in some way. 
Should the teacher ignore his answer altogether, it is not considered 
a response opportunity. 

The public nature of the "response opportunity" distinguishes 
it from the various foms of teacher-afforded and child-created dyadic 
contacts (procedural, work-related, and behavioral). In the teacher- 
afforded and child-created work-related contacts, the teacher talks 
to' the child about his own individual seat work. Teacher feedback 
here is "private," meant only for the child involved and not for the 
class as a whole. These contacts occur when individual children bring 
their work to the teacher to ask him about it or when the teacher goes 
around the room correcting work individually at each desk. It fre- 
quently happens that the teacher will question a child when dealing 
with him individually about bis seat work. Such an event is coded 
under work-related dyadic contacts and is not considered a "response 
opportunity," since the question is meant only for the particular 
child involved and is not a public question. 

Response opportunities must also be distinguished from reading 
and recitat'on turns, which are not coded in this system. The major 
distinction is that response opportunities are initiated by a teacher 



question which requires a focal » circumscribed answer* Reading and 
recitation turns are more extanded performances by the child, In which 
he responds at length to an Initial question or command* Ordinarily 
these will Involve verbal demonstration of mastery (overlearnipg) of 
skill, as when reading aloud in reading groups or reciting mathematics 
tables* Response opportunities involve focal questions 

which, along with the answer given by the child and the ensuing feed- 
back, form a natural unit* Each such question-answer- feedback segment 
constitutes a self-contained interaction sequence in its own right, 
easily separable from preceding or following units, even when they 
involve 'the same child. Whenever the response demand on the child 

is such that he will continue responding until and unless he makes a 

mistake, the interaction is a reading or recitation turn and not a 

response opportunity and therefore should not be coded. 

Each response opportunity which is coded requires the checking of four 
separate bits of information: the type of response opportunity, the level of 
question asked, the quality of the child's answer, and the nature of the teach- 
er's feedback response. The last item to be coded (teacher's feedback) some- 
times will be conqplex^ enough to include two or more of the categories of 
teacher fesdback, so that some response opportunities will require five or 
more separate markings. 

Four types of response opportunity have been identified: In the first 
type, the teacher names the child first and then asks her question. This 
column on the coding sheet is labelled (PRE); in the second type the teacher 
asks her question first, but she calls on a child who does not have his hand 
raised or a non-volunteer (NVOL). The third type of response opportunity in- 
volves the teacher's asking a question publicly but calling on a child who 
does have his hand raised (volunteer or VOL)* The fourth type of situation is 
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the call out (CALL) . 

Response opportunities cseated by children who call out answers 
to teachers' questions without waiting for permission to respond are 
coded In the call out column. The teacher creates the response 
opportunity by asking a public question, but one child calls out an 
answer to this question before he has a chance to Indicate that a 
particular child should respond. This type of response opportunity 
Is therefore child-created, In that It was not the teacher's Intent 
that the child answer the question. Besides those already mentioned, 
one additional consideration must be present before coders code a 
response opportunity under call out ; the teacher must recognize the 
child's response and make sone response to the child In reaction to It. 
Called out answers which are Ignored b;^ the teacher are not considered 
response opportunities and are not coded . A response opportunity 
coded as call out then, requires the following: (a) the teacher asks 
a public question; (b) the child calls out an answer to the question 
before the teacher has a chance to call on anyone to respond: (c) the 
teacher then turns his attention to »the child who called out the answer 
and says something In response to him The teacher's response to the 
child oust contain feedback regarding hlsanswer to the question; the 
Interaction Is not coded as a response opportunity under call out If 
the teacher confines her remarks to criticism of the child for calling 
out the answer. It Is necessary, therefore, that the teacher make 
some feedback response to the child who calls out the answer. 

Just as there may be confusion In distinguishing between questions 
directed to a non volunteer and questions directed to a volunteer when the 
coder is unsure whether or not the child has raised his hand, there may 
also be confusion In distinguishing call outs if the coder Is unsure 
whether or not the teacher made seme Indication to the child that he 
should answer the question. There is usually little problem when the 
teacher calls jn the children by name, but some teachers will call on 
children by pointing at them or othepwise non-verbal ly indicating that 
they should make a response. Coders should be particularly 



alert with such teachers to pick up these less obvious cuea g en to children 
to signal their permission to respond. When the coder is oot sure vhetbf^r or 
not the teacher made such a signal , anc therefore is not sure whether or not 
to code a question to a volunteer (VOL) or a call out (CALL)^ the interaction 
should be coded as a call out . 

Similarly 9 when the coder is not sure whether the child selected had his hand 
upt VOL should be coded. 



6 



I£VEL OF QUESTION 

After noting the type of response opportunity and the identity of 
the child involved by entering the child's number in the appropriate 
column, the cojer row codes the level of question asked by the teacher. 
Level of question refers to the nature of the rcsporse demand made 
upon the child. Thrac levels are identified: process questions, 
product questions, and choice questions* 

These three levels refer only to questions about academic or 
school-related content. . 

To determine the level of the response demand built into teacher's 
questions the coder must make two decisions: (a) he must decide whether 
the question is an academi-: question or a self-reference question: (b) 
if it Is an academic question he must decermine v^ether it is a process 
question, product question, or choice question. Academic questions concern 
factual matters connected with curriculum content of the school. They 
require the child to make a response showing that he has certain knowledge 
of information, to provide such infoxpation himself in answering the ques* 
tion, or to explain something at length showing his grasp of the principles 
involved. The content of the question deals with reading, writing, arith- 
metic, social studies, science, spelling, or otheraspects of curriculum 
which the school is attempting to deliberately teach the child. Questions 
dealing with these matters are considered academic questions and subdivided 
into process, product, and choice questions. Questions that do not deal 
with such factual matters but instead ask for the child's preferences, 
personal experienced, ahd so forth are tallied in the boxes under SelN 
Reference questions. Questions which deal with a child's opinions or 
predictions are coded separately as Opinion Questions , Both the Self- 

RcferftPfif and QpIgiQn categories will be described later. 
Process Questions 

This is the most complex level of question, in which the child 
is required to explain something in a way that requires him to inte- 
grate facts or to show knowledge of their interrelationships. It 
most frequently is a "why?" or "how?" question, and usually requires 
an extended phrase or sentence for fo^rmulating an adequate response 
single word answers are not usually sufficient. A process question 
requires the child to specify the cognitive and/or behavioral stops 
that must be gone through in order to solve a problem or come u£ with 
an answer. 



BxMples: What can we learn fros this story? 
What does that saying mean? 
Why should ve not play with matches? 
Hov do new planes g^ov from old ones? 
Why does it get dark at night? 
Hov do you know that that's a long "e" sound? 

Why is that a wrong answer? 
What should you do if ... ? 

As always, the teacher's intent determines the coding. For example, 

the teacher may ask "When you ride your bike. and come to a stop sign. 

what do you do?" Ordinarily this would be coded as a product question 

demanding the answer "Stop." However, If the question appears just after 

a lesson in which the teacher had explained the process of stopping 

(stop the bike, carefully look right and left, judge the distance of any 

cars In sight, and quickly get to the other side, etc.), this question 

would be coded as a process question . Thx^ example illustrates the 

procedure t> be followed when In doubt in determining whether a question 

should be process versus product, if the teacher seems to be requiring 

a process answer, that is a long explanation of a complex sequence of 

events, process question ihould be coded. If on the other hand he seems 

to be satisfied with a simple St. -t Answer, product question would be 

coded . 

Product Questions 

Product questions seek a specific correct answer which can be ex- 
pressed in a single word or short phrase. They do not involve the 
explanations built into process questions, and at the same time they 
do not provide the child with alternatives which include the correct 
answer, as in choice questions. Thus the child must either know the 
answer and verbalize it or take a guess by encoding an answer on his 
own. 

Examples: What (letter, number, day, shape, color, etc.) is this? 
Who (discovered America, is the president)? 
What is this? 

When (is Christmas, was America discovered, etc)? 
Where (is Boston, do we buy food, etc.)? 
What do we get from cows? 

How many ^ are there? 

How do you spell ? 

What do buses do? 

What is this %9ord? (a question requiring the child to read 
a single word is coded as a product question rather ti:an as a reading 
turn, which Involves reading at length) 
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Product; questions usually 6egin with 'Hiho?", •'what?", 'Hihen?", 
••where?", "how ouch?", or "how many?". Many of the response opportunities 
in the early school grades will be coded as product questions if the child is asked to 
identify a letter, produce a sum or remainder, etc. While the child 
may have to go through many cognitive processes in order to arrive at 
the answer, the question itself as asked does not require him to 
verbalize these processes but only to produce the answer* So long 
as this is true the question is a product question , and the response 
demand on the child is less than it is for a process question, since 
less is required of the child and since th3 possibility remains that 
he might guess the answer without knowing the process that the teache^ 
wants him to know. 

The following example occurred during a reading group: The teacher 
gave each child a card with a word on it and then told the children, each 
in turn, to read their word and then place it under the picture that it 
matched. This was coded as two separate response opportunities for each 
child; the first one being a product question (read the word), and the 
second being a choice question (match the word to one of the pictures) . 

In. discussing stories or pictures there sometimes will be difficulty 
in dstlnguishlng product ques.tions from self- reference questions. As 
always, coding must follow the teacher's apparent intent. Thus if the 
answer to the question is ::o be found by examining the picture (What 
color is Splly's wagon?), the question is coded as a product question . 
On the other hand, if the teacher is not asking for a factual answer but 
wants to get opinions on what the children think might happen (What's 
Dick going to do now?), an opinion question is coded* In general, 
if the teacher is fishing for the right answer he is asking a product 
question; if he is instead only trying to get the children to express 
Themselves or to talk about the picture, self -reference or opinion 
are coded. Sometimes the teacher will begin with a product question and, 
seeing that he isn't going to get the answer, will continue to ask various 
children what they think will happen, etc., so that the remainder of the 
questions will be coded as self- reference opinion questions. 



choice Questions 

In the choice question the child does not have to produce a sub- 
stantive response but may inst€ad simply choose one of two or more 
implied or expressed alternatives. Included are yes-no questions, 
either-or questions, and questions which present more than two alternatives 
but which make it clear that the correct answer is one of the alternatives 
presented . Choice questions are of interest because they tend to en- 
courage guessing by maximizing the child's chances of producing correct 
answers (response products), even though he may lack the correct know- 
ledge or skill (response process) that the teacher assumes to be 
operating when children answer correctly. Choice questions in^-olve a 
more limited response demand upon the child than do product questions, 
since unlike the latter they do not require the child to produce a 
substantive response on his own; the child knows that the correct 
answer is one of the alternatives the teacher presents in asking the 
question, and If he is disposed to guess he can make a response by 
indicating one of those alternatives. Occasionally a large number of 
alternatives will be present, as when the teacher asks the child to 
indicate or underline one particular letter of the alphabet (out of 
the 26). This nevertheless is still coded as a choice question 
because the child knows that the correct answer is one of the alter** 
natives presented. When the alternatives are presented verbally, 
th ere are usuall y only two or three alternative cateeories of response. 

Two criteria distinguish choice questions: (a) the question deals 
with academic content and cannot be classed as a self -reference ques- 
tion; (b) the teacher provides response alternatives, either verbally or 
by showing the child visual aids to look at in connection with the ques- 
tion, which include the correct answer among them (ie.:, the correct 
answer is one of the alternatives presented). Examples: 

Is this (b or d, 3 or 4, Monday or Tuesday, a square or a circle, 

red or blue)? (either-or questions) 
Which of these is (taller, smaller, blue, a vowel, the same as this 

one, etc.)? (select the right answer from among the altema* 

tives presented) 

Are these (the same, blue, circles, synonyms, correct, etc)? (Yes- 
no questions) 

Which four of these five things go together? (the child must pick 
four pictures but nevertjieless the correct answers are pro- 
vided in the alternatives sho\m) 

The big bear sat on a bro%m box. Which words start with the same 
letter? (although more difficult, this is still a choice 
question in that the alternatives are provided in the ques- 
tion itself) 
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Look at the color words on the black board. Which ones start with 
the letter ''b*'? (Again, the correct answers are included in 
the alternatives presented. If instead the children were 
expected to pull these from memory (What color words start 
with the letter ••b"?) without any reference to concrete exam- 
plea of color words, the question would be coded as a product 
question. ) 

Make an X on all the animals that have a tail. (Any workbook or 

worksheet exercise which involves marking one or more of a set 
of alternatives according to some rule is treated as a choice 
question, since all the alternatives are provided.) 

Coders should bear in mind that any question which is an either^or 

question or a yes-no question is coded as a choice question , regardless 

of the complexity of the content. Examples: 

If I pour the water from this white dish into this test tube, will 
there be more water, less water, or just the same amount? 

Are the lines of a rectangle equal and parallel, equal but not 
parallel, or parallel but not equal? 

Which is better to put out a grease fire wacer or sand? 

Although the preceding examples are apparently complex, it neverthe* 
lesa remains possible for some children who do not understi^ad the processes 
involved to be able to respond to the question, since the response alterna- 
tives are provided in the question ttself. Thus should the child decide to 
respond rather than say that he doesn't know or ask for more information, 
he can respond by verbalizing one of the response alternatives back to the 
teacher. 

Sometimes a question which would ordinarily be classified as a product 
question is coded as a choice question because of the immediately preceding 
events. The previous example "What color words start with 'b*?", for instance 
would be classified as a choice question if the teacher had preceded it by 
calling the children's attention, to concrete examples of color words (by 
writing them on the board, showing visual aid materials on which the color 
words were printed) . Another example occurred in the science lesson in 
which the teacher gave an extended presentation about how leaves could be 
classified according to size, shape, and color. She repeatedly compared 
pairs of leaves explaining that she was looking for similarities and dif- 
ferences in size, shape, and color. The repetitive nature of her pref;enta- 
tion and the restriction of her language to the key words "size," "shape," 
and "color" led evei.tually to the isolation of these three words as a 
restricted set of alternatives to respond to the question 'How are these 
two leaves different?" When she later began asking the children to com- 
pare leaves her questions were coded as choice questio ns, since she had 
identified and reinforced "size," "shape, and "color" as the response 
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alternatives she had in mind and because she accepted with apparent 
satisfaction the responses of children who simply verbalized one of 
these key words without any additional material. 
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CHILD'S ANSWER 

After coding the child's identity, the type of question, and the 
level of question, the coder notes the child's answer into one of four 
categories: correct, partially correct, incorrect, or no response . 
The teacher's intent is taken into account in determining the correct- 
ness of the child's response. Frequently teachers may ask ambiguous 
questions which are answered correctly or partially correctly from 
one point of view but which are treated as incorrect by the teacher, 
who was looking for a very specific answer. Thus it is the teacher's 
perception of the correctness of the child's response which is coded, 
not the coder's perception. This distinction is important because the 
next variable coded is the teacher's feedback to the child's response, 
and this feedback is considered to be feedback to the child's answer 
as perceived by the teacher* Consequently if the teacher reacts to a 
response as if it is wrong it is coded as wrong, even though anothet 
observer might consider it to be partially or even completely correct. 
Correct Answers 

If the child answers the teacher's question in a way that satisfies 
him, the answer is coded as correct. Determination of whether or not 
the teacher is satisfied with the child's answer does not necessarily 
raquire that the teacher positively affirm the answer or make some 
favorable response to it. Instead, the child's answer should be 
considered correct unless the teacher makes some positive action 
suggesting dissatisfaction with it (explicitly explaining that the 
child's answer is incorrect or only partially correct, giving the 
"correct" answer, or asking someone else to answer the same question). 
If the teacher does not make an attempt to improve upon or replace 
the child's ansver with another, his answer is considered correct. 
This means that some answers that the coder would not accept but which 
the teacher treats as correct are to be coded as correct a-^swers. 
Part-Correct Answers 

Part-correct answers are answert which are correct bu^ incomplete 
as far as they go or answers which^^are correct from one point of view 
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but not the answer that the teacher is looking for. Again, the teacher's 
feedback response may determine the way the answer is coded. If the 
teacher indicates that the child^s response is correct but incomplete, ^ 
or if he indicates that the response is correct or defensible but not 
the answer that he is looking for, code the response as part-correct. 

1. An answer is coded as part correct whenever the teacher indicates 
ambivalence about the response. This means that the teacher may accept 
the response as correct as far as it goes but note that it is incomplete 
(«8 when the child gives only one part of a two part answer);- another type 
occurs when the child's answer is more specific or more general then the 
particular one that the teacher had in mind, so that the teacher must 
indicate both the validity and the imprecision of the child's answer 
(''Well, it is an animal, but what kind of an animal is it exactly?"). 
Part correct answers will be coded most frequently when the child ;.ro<- 
duces an answer that the teacher had not anticipated. Often this will 

be because the teacher's question was more ambiguous than the teacher 
realized when asking it. 

2. Sometimes the child will make;^ an answer that is correct in content 
but is not presented in a form which satisfies the teacher. Examples 
include shaking the head to indicate "yes" or "no" rather than responding 
verbally, answering the question in a word or a phrase when the teacher 
wants it put into a comp.Iete sentence, counting on the fingers when the 
teacher wants the .Mid to do the problem in his mind, etc. These answers 
are also coded as part correct, since the teacher accepts the correctness 
of the content but criticizes the form. 

Incorrect Answers 

Responses coded as incorrect answers are those in which the child's 
response is treated as simply wrong by the teacher. The teacher need 
not explicitly tell the child that he is wrong; he may indicate this 
Indirectly by searching for the answer from someone else or by pro- 
viding it himself. In one of these ways the teacher indicates that 
the child's answer is not an acceptable response to the question he 
has asked. 
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Don't Know 

Mumbling vhich does not appear to be an attempt to answer the question , as when 
the child seems to be talking to himself or perhaps membling "I don't know," 
would be coded as don't know , <DK)« 

This category is included in the coding system specifically for those 
Instances when the child clearly does not answer the question which the 
teacher puts to him^ and^ in effect, says so, or makes some verbal response 
indicating this. . 

No response is coded whenever the child remains silent. If the child 
does make an intelligible response to the question it must be coded as correct 
part correct y or incorrect. Thus if a child mumbles an answer to a teacher 'r: 
question and is asked by the teacher to repeat his answer more loudly, the 
answer will be coded as either part correct or incorrect, depending on the 
reason the teacher asked the child to repeat the question. If the teacher wants 
the child to repeat because she has heard his response but wants the other 
children to hear it or wants to avoid allowing children to mumble responses, 
the child's answer is coded as part correct, in that it is acceptable content 
delivered in unacceptable form. On the other hand, if the teacher is asking 
the child to repeat because the teacher has been unable to hear the child's 
answer and does not know whether it is correct or incorrect, the child's 
answer is coded as incorrect. Any mumbled answer which apparently is an 
attempt to answer the question is treated an incorrect as long as it remains 
unintelligible. 

To summarize: if the child attempts to answer the teacher's question, his 
answer is coded as correct, part correct , or incorrect , depending on the teacher's 
reaction to it; if he indicates that he is unable to answer, it is coded as 
don't know (DK) or If he does rtot attempt to answer the question, it is coded 
as no response (NR). 
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SYMBOL FEEDBACK REACTION 

++ Praise (positive evaluation) 

Criticism (negative evaluation) 

0 No feedback response — teacher does not react to 

child's answer 

Pc88 Process feedback 

Giv Ans Gives correct answer (without getting into process) 

Ask 0th Asks another child to give the answer 

Call Call Out (some other child calls out the answer before 

the first child responds to the question) 

Rept The teacher repeats the question 

Reph or Clue Teacher rephrases Jhe question or gives a clue 

New Q Teacher asks a new question 



The first seven of the ten categories listed above are desig- 
nated as "terminal" feedback, while the last three are called "sus- 
taining" feedback. This is one of the key distinctions involved In 
studying communication of teacher Acpectations . The categories of 
sustaining feedback include teacher behavior which prolongs the response 
opportunity by providing a second chance to deal with the same or 
related questions. Use of sustaining feedback reactions is an index 
of the teacher's willingness to stick with the child until he can pro- 
duce an acceptable answer. Terminal feedback, on the other hand, brings 
the response opportunity to a close. With terminal feedback reactions 
the teacher either gives the child the answer or sees that he gets it 
from someone else, or merely makes a feedback or evaluation lesponse 
without supplying the answer. In e ither case, he does not s stain 
the interaction and provide additional response opportunities. 

The terminal feedback categories may also be profitably sub- 
divided for some purposes to the first three categories, which do not 
involve a substantive response or answer, and the second four 
categories, which do involve such an answer. The ten categories, then, 
may be summarized as follows: the first three categories of terminal 
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feedback provide the child only with positive or negative evaluation, or 
feedback^ and not with substantive information; the last four categories 
of terminal feedback do provide substantive information to the child, 
either from the teacher or from one of the other children; the final 
three categories (sustaining feedback) provide the child with a 
second response opportunity, either to answer the same question or to 
answer a related one. Hie categories are defined so as to be mutually 
exclusive but not contradictory, so that more than one category may 
apply to a given teacher feedback reaction. In such cases, each new 
category of teacher feedback la simply noted in the order in which it 
occurs. Certain types of multiple-category teacher feedback reactions 
require special coding conventions, but discussion of these will be 
deferred until the categories themselves are presented in more detail. 

Praise 

Praise refers to the teacher's evaluative reactions which go 
beyond the level of simple affirmation or positive feedback by verbally 
complimenting the child ("Good," "Fine," "Wonderful," etc.) and/or by 
accompanying verbalization of positive feedback with expressions or 
gestures connoting excitement or warmth. Thus praise is coded when 
• the teacher does something more than merely indicate that the child 
has given a correct response. He communj jates a positive evaluation 
cr a warm personal reaction to the chili and not merely an impersonal 
communication of information. 
prltlciBm 

Criticism parallels praise In that it refers to negative teacher 
evaluative reactions that go beyond, the level of simple negation by 
expressing aiiger or personal criticism of the child In addition to 
indicating the incorrectnet «i Ci. nia response. The' category includes 
obvious verbal criticism <'tha;''j< a stupid answer/' •'What's the matter 
with you?" ••if you'd pay attention, mayb^ you'd get it right") and 
verbal negation which is accompanied by expressive or gestural communi- 
cation of hostility, anger, disgust, or sheer frustration. In general, 
any verbal response which disparagingly refers to the child's in- 
tellectual ability or, more frequently, his motivation to do good work, 
it coded as criticism . Statements of latter type by the teacher may 
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b« factually trua (l.a., tha child nay not hava baan paying 'ttantlen) 
or nay ba unvarl£labla gratultu)ui rajectlon ("You Juit don't care"). 
Both ara never tha laii coded criticism, §f.nct thli coding re f era 
to the teacher 'i behavior £er ae and not to tha veracity or Juatifica- 
tion for hie aratementa. 
No Feedback Reaction 

If the teacher makes no response whatever following the child's 
answer to the question, he is ceded for no feedback reaction. This 
means that he makes no verbal response to the child and does not 
communicate affirmation or negation by shaking his head in response 
to the answer. Instead, he merely moves on to something else, perhaps 
by starting to make a new point or by asking another child a question. 
Most coders will be surprised to find that this category is used much 
more often than they had expected. It frequently happens that the 
teacher makes no feedback reaction at all to the child's answ- , 
especially In fast-moving question drills where he is pushing to get 
correct answers in an impersonal fashion, without paying attention 
to the individual child giving the ahswer. 

In addition to the obvious condition of no feedback reaction 
o tlined above, whsre the teacher says and does nothing in reaction 
t_ the child, one special type of teacher reaction is also coded in 
this category. This occurs when the teacher repeats the child's 
anbwer in a quizzical manner without indicating whether he considers 
It to be correct or incorrect. This reaction may frequently occur 
when the teacher is asking the children to guess, give opinions, or 
make predictions about something. In such instances he may reply to 
the child's answer ("He's going to go home and tell his mother") with 
an ambiguous response ("You think he'll go home and tell his mother?"). 
Unless the teacher's . sedback reaction Is furt.ier elaborated to pro- 
vide affirmation or negation or some substantive answer to the child, 
It Is coded as no feedback reaction. 

Proceaa Feedback 

Xh» process versus product distinction introduced previously in 
dlacussing level of question is also^used in coding the level of 
teacher feedback. Process feedback is coded in the present category, 
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while the following three categories refer to product feedback 
(simply giving the answer). Process feedback is coded when the 
teacher goes beyond merely providing the right answer and discusses 
the cognitive or behavioral processes that are to be gone through in 
arriving at the answer. In other words, he reviews the question or 
problem with the child at length, telling him hov to go about respond- 
ing to it and not merely what the correct answer is. Process feed- 
back occurs most frequently following errors, when the teacher ex- 
plains the reasoning processes to be gone through to arrive at the 
correct answer or explains the erroneous processes followed by the 
child to arrive at the wrong answer. Process feedback may sometimes 
fjllow correct answers, as when the teacher elaborates jn the response 
to verbalize the process knowledge it represents ("Yes, we kno r that 
we should use a capital letter since it ^ !^ proper name, and all 
proper names begin with capital letters"). Teachers may providp 
process feedback by simply answering a process question, since by 
definition a process question requires a process answer. Other than 
this special situation, however, process feedback will usually require 
elaboration upor rlie answer to a question. 
Gives Answer 

This category is used \.hen the teacher gives the child the answer 
to the question, but does not elaborate sufficiently to be coded for 
process feedback. The category is used only wh^n the child has given 
a wrong answer or has not answered the question. When the teacher gives 
an answer to a process question it is coded as process feedback. Other- 
wise, any situation in which the teacher promdes the answer to the 
question to vhich he has asked is coded as gives answer . Usua? ly this 
will correspond to product feedback following product questions, 
although occasionally giving the answer to choice questions may also 
be coded here if the child does not take a guess and try to answer 
the question himself. 
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Asktt Other 

If the teacher does not answer the question himself but instead 
asks fiome ether child to answer it, the fftudback is coded as asks 
other. This category is coded regardless of the level of question or 
feedback involved (i.e., feedback to process questions is still coded 
under asks other if the teacher asks another child to provide the 
/ingwer). Sometimes the teacher will ask another child very explicitly 
to anjwer the question that could not be handled by the first ("Johnny, 
can you help Mary?").^ However, this need not b« ao explicitly stated 
for as ks other to be coded. Whenever tho child dues not answer a 
teacher question and the teacher moves to another child in order to 
get the answer to that same question., the teacher's feedback reaction 
Is coded for as ks other. 
Call Out 

The call out category is used when another ch* M cai:.-, out the 
answer to the -luestion before the teacher has a chance tu act on his 
own. This category is coded regardless of thp level of question asked: 
if another child calls out the answer to the teacher's question befort- 
either the first child or the teacher himself can provide fhat answer, 
the feedback category call out is coded. Usually this will mean also 
cod.ng a response opportunity for the child who called out the answer, 
provided that the teacher makes some individu^I response aifter he calls 
out the antrfer. In »ny case, the feedback coded for the first child is 
c all out. 
Repeat s Question 

This category and the two tc follow comprise the categories of 
sustaining feedback, in which the teacher sustains the response oppor- 
tunity and provides the child with a secor.:? chance to respond. The 
first such reaction is when the teacher simply repeats the question . 
This will almost always occur wheu the child has made no response, 
although it may also occur at times in v^hich he has given an incorrect 
response. In any case, if the teacher asks a question, waits 3ome 
time without getting the correct answer, and then repeats tho questina 
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same child , his feedback reaction is coded as repeats question . 
The teacher need not repeat the entire question word for word in order 
to be coded in this category. Truncated versions of the original 
question and short probes to determine if the child can make any 
response to the original question, are both coded as repeats question . 
For example, to the original question 'Wiat color is this?" the follow- 
ing responses are all coded as repeats question ; '"What color?" 
?Vell?" "Do you know?" "John?" (The latter said in a manner that 
comnunicates that th^ teacher is waiting for the child to respond to 
his original question). 

In each of the variants mentioned above, the teacher is coomunica-^ 
ting that he is waiting for the child to respond to the original ques* 
tion and that he still v^ants him to respond if he can. The teacher 
does not change the question, as in the following categories, but 
merely repeats it or refers to it as it was asked previously. 
Rephrase or Clue 

In this feedback reaction, the teacher sustains the response 
opportunity by rephrasing the question or giving the child a clue as 
to how to respond to it. Usually the rephrasing of the question in 
this situation will be such as to simplify it, parcicularly in moving 
from a product question ("Vhat color is this?") to a choice question 
("Is it red or blue?"). Rather than rephrase the question in this 
manner, the teacher may provide a clue expressed as a declarative 
statement; "It*s the same color as an apple." Two key considerations 
determine the coding of rephrase or clue in teacher feedback: (a) the 
teacher does not merely repeat the question as originally asked but 
embellishes it in some way to make it easier for the child to respond; 
(b) nevertheless, he is still seeking the same response as asked for 
in the original question. The latter condition separates the present 
category from the category of new questions which follows, in which 
the teacher asks a new question which requires a different answer from 
the one asked originally. 
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The material provided by the teacher In rephrasing the question 
or giving a clue may or may i>ot be helpful for the child certain 
types of clues may actually confuse him rather than help him. This 
£act should not be allowed to Influence the coding. So long as the 
teacher does something which is Intended b^ the teacher to help the 
child answer che original question , the teacher's action is coded 
as rephrase or clue. 
Mew Question 

The teacher asks a new question when she requires an answer that 
is different fron the 6riginal question, although it may be closely 
related. A question requiring a new answer is coded as a new question . 
This is the only criterion. Thus to the original question 'Vhat 
color is .this?", questions which elicit the same answer (''Is it red 
or blue?" "Is it red?") are coded as rephrase or clue . Questions 
which seek to elicit a different answer are coded as new questions 
("Well, %ilmt color is this one?" ''Have you been studying your home* 
work?" "Is it bright or a dull color?"). 

The occurrence of sustaining feedback (repeats question, rephrase 
or clue, or new question) presents a soeclal coding problem because 
this type of feedback gives the child a new response opportunity . 
This new response opportunity must then be coded for level of question, 
quality of answer, apd additional feedback from the teacher. At the 
same time, the fact that it is a follow up to an original response 
opportunity rather than a wholly new response opportunity must be 
maintained in the coding system. This is accomplished by skipping 
down to the next row \Aenever sustaining feedback is coded, thereby 
bringing a close to the coding of the original response opportunity 
and beginning the coding for the follow up response opportunity. On 
the next row the level of question, the quality of the child's answer, 
and the nature of the teacher's further feedb<%ck is coded. Follow-up 

response opportunities occurring due to sustaining feedback in reaction 

to the original response opportunities are coded for type of response 

opportunity, which would be coded non-volunteer (NVOL) in all cases 

of sustaining feedback, level of question, quality of child's answer, 

and type of teacher feedback. 
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OtOier than the special Qpnditions requiring skipping to a new 
rowiAen sustaining feedback occurs, the coding of teacher's feedback 
reaction simply involves noting the appearance of new codable feed- 
back categories 

Note also that two or more occurrences of the same type of 
sustaining feedback (repeats question, rephrase or clue, or new 
queatioxp may occur in succession and be coded separately. Thus a 
teacher might repeat the original question (or make some attempt to 
get the child to answer it) two or three times rather than just once. 
In such a situation, each repetition of the original question is coded, 
so long as there is some time in between ^ich amounts to a new 
response opportunity being extended to the child. However, redundant 
repetition of the question CVell — do you know?!) is coded as only a 
single repetition since no time for an opportunity to respond is 
allowed between parts of the question. When such time is allowed 
(••Well? ... Do you know?"), two separate repetitions of the ques- 
tion are coded. 4 
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APPENDIX: 

Examples of Teacher^s Feedback Reactions 



To facilitate comparison of examples of teacher feedback reactions to 
the answers of the children, examples will be given vith reference to three 
typical teacher questions and child answers. The thr«ee situations are as 
follows: 

Question one: What color is this? (the correct answer is ''Red") 

Question two: Uhat*word is this? (the word is "Bad") This question 
might be asked as stated or might be implied during the reading group, as 
when a child is reading but gets stuck when encountering the word "bad''. 

Question three: How do you think John feels? (the answer is "Bad" 
or any one of its synonyms) 

Examples of teacher feedback reactions which might be made to the 
child's answers (or failures to answer) to the previous questions are 
presented below. Under each heading the feedback reactions following the 
number 1 refer to reactions to question one; those following the number 2 
refer to reactions to question two; and those following the number 3 refer 
to the reactions to question three. Additional material and discussion 
ot special situations will appear after the examples for each of the 
twelve categoties of teacher's feedback reactions. 
Praise 

1. "Red!" (delivered with gusto and warmth) 
"Right — it's red. Good, Johnny." 

"Good." (said in response to a child who has given the correct 
answer) 

"Yes, you really know your colors, don't youf." 

2. "Good — you remembered didn't you!" 
"Bad! Very good, Johnny." 

"Right — you figured that out all by yourself didn't you!" 

3. "Yes, I think you're right » Johnny, that's good thinking." 
"Right, Kary! You read the story and found out how Johnny 
felt, didn't you?" 

Criticism 

Teacher feedback reactions coded as criticism include negation accom- 
panled by gestural or expressive communication of anger, rejection, or 
frustration as well as direct verbal criticism: 
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••Kaybe you*d know if you'd pay attention." 

"You vouldn't make mistakes like that if you tried harder." 

"Don't guess look at the word. You should knov better than that." 

"I told you to raise your hand before answering — » weren't you 

listening?" ^ 
"We've been over this three tines already, John — ^ you should know it 

by now." 

"That's not right — what's the natter with you?" 

MO Feedback Reaction 

The teacher is coded for no feedback reaction if he sinply does not 
respond to the child following his answer of if he makes a verbal response 
which does not communicate information about the correctness or incorrectness 
of the child's answer. Examples of the latter: "You think it's red;" 
"I never thought of that." 

Process Feedback 

1. Proceas feedback iB notrpossible in reaction to the child's answer 
tc the first question, since the question deals with the arbitrary Unguis* 
tic label which the English language attaches to the color "red." These 
and equivalent questions involve basic facts which must be simply memorized 
rather than explained. Since the correctness of the correct answer resides 
In arbitrary societal consensual agreement rather than in the presence of 
a logically based sequence or process, no process feedback is possible. In 
addition to color labels, other categories of questions which do not 
admit of process feedback include spelling, traffic signs and turn signals, 
and the interrelationships among units in systems of measurement. Thus 
process feedback could be given to a child when the question involves tel- 
ling tine from the clock, but not when the question concerns the number of 
minutes per hour or the number of hours per day. 

2. Johnny, in order to read the word you have to sound it out 
(followed by a demonstration of how to sound out the word). When you 
don't know the word you can sometimes figure it out by thinking about the 
story so far and by looking at the picture (followed by an extended 
explanation of how the child might have figured out the word was "bad" 
by figuring out that Johnny felt bad in the story and that the particular 
sentence was describing how Johnny felt). 
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3» To figure out how Johnny feels you have to think about the story 
and about what happens to him (followed by a discussion of significant 
events in tha story which would suggest that Johnny feels "bad"). 
Gives Answer 

1. It's red- We call this color red- It's red, just like a 
stop light. 

2- Bad- The word is bad. B*A*D spells bad. Not bed bad. 

3. I think John probably feels bad. He doesn't feel very good, does 

he? He is very unhappy, (assuming the teacher equates this with 

"bad") He feels awful. 

Asks Other 

Here the teacher does not provide the answer for the child but instead 
asks for someone else to provide it: 

Does anyone know? 
Mary, can you tell me? 
Cm someone help John? 

What is it, class? (the teacher may call for a chorus response rather 
than ask for a single child to respond) 

Call Out 

Call out is sometimes coded for the teacher's feedback reaction (al- 
though it is not a teacher response) if some other child calls out the 
correct answer when the first child gives an incorrect answer or is unable 
to respond- This includes both instances in which the child who calls out 
the answer is coded for response opportunity (because the teacher then 
turns his attention to him and makes a feedback response) and instances in 
which the child who calls out the answer does not get coded for a response 
opportunity (the teacher does not turn his attention to him and give 
specific individual feedback). Thus call out has a slightly different 
meaning for purposes of coding teacher feedback reaction than it does 
for coding response opportunities for individual children. Call out is 
coded in teacher's feedback reaction whenever the child gets feedback 
from another child who in fact calls out the answer; it is not necessary 
that the teacher give feedback to the child who called out the answer. 
Repeats Question 

1- What color? Well? Do you know? 

2- Do you know that word? Are you stuck? What is it? 
3. How does he feel? what do you think? Hmnmn? 
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Rephrase or Clue 

1. Is it red or blue? Is it red? Is it blue? It's the same color 
as a stop light. It's our new color for today* It begins with "r". It 
rhymes with ''bed", 

2. Is it bad? Is it had or bad? Does he feel good or bad? Look 
at the first letter. What word does it rhyme with? We just had this 
word up here (pointing). How does Johnny feel? He feels ? 

3. Does he feel good or bad? Does he feel bad? Well, is he happy, 
sad, angry, or what? Look at his face. He's never going to see Sam 
again. How would you feeJL if you were Johnny? How does he look? 

New Question 

1. Yes, and what color is this? What else is red? Are you wearing 
anything that's this color? 

2. Why did he feel bad? Is he crying? Did you study this story? 
How do you spell that word? 

3. And how does Sam feel? Yes, how could you tell that he was sad? 
Then what happens? Why does he feel sad? 

In general, the teacher's feedback to the child is coded as :>rocess 
feedback if he explains why an answer Is wrong or if he explains what to 
do in order to get the right answer. If the original question was a 
process question, the teacher will be giving process feedback simply by 
giving the answer to that question. This includes the extreme case in 
which the child has answered the question correctly and the teacher re* 
sponds merely by repeating the child's process answer. Except for the 
special case of process questions, however, the teacher must go beyond 
simply giving the answer to the original question in order to get credit 
for process feedback. For example, the teacher may be observing a child 
writing his name on the board. If she merely says "No, Johnny, you 
put a little 'J', your name begins with a capital 'J'." she would be 
coded for Ji^roduct feedback . However, if the teacher explained about 
names being proper nouns and proper nouns always being identified with 
an initial capital letter, she would be codad for process feedback . 

The teacher may sometimes be credited with process feedback when 
this feedback is apparently not understood and therefore not successful. 
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The key consideration, however, is «n stteapt to coinminicste to the child 
vhy his response was wrong and y> help hia understand the processes 
involved, and not necessarily the child's success in reaching this under- 
standing. Consider the following example: 

Teacher: What color of clothes should you wear when riding a bike 
at night? 

Child: Red, or maybe white. 

Teacher: Don't you think you might want to wear white so that you 
could be secP better? 

The teacher in this feedback reaction attempts to coomunicate the 
rationale underlying the choice of white as the appropriate color. This 
may or may not be understood by the child. The teacher is nevertheless 
credited with process feedback because of his attempt to delineate the 
rationale. 

Differentiation among repeating the question, rephrasing the question, 
and asking a new question requires consideration of both che teacher's 
apparent intent and the response demand of the second question. For instance, 
when a child is reading and stops becfiuse he apparently does not know the 
next word, the teacher reaction "Are you stuck?" can be seen as function* 
ally equivalent to "Do you know the woxi?" and therefore codable as repeat . 
However, the reaction "Did you study this?" is different. Here the 
teacher is not merely inquiring about whether the child knows the word or 
wishes to make a guess. He has shifted focus to the more general matter 
of the child's reading ability and faithfulness in practicing it. 
Consequently, this reaction is coded as a new question , since it demands 
a new response and is not an attempt to get the child to produce the word. 
The teacher, reaction "How does Johnny feel?" %iould be coded as repeat with 
with reference to question three of the examples. However, its cppearance 
in connection with question two, when the child was stuck when trying to 
read the i#ord "bad", would be coded as providing a clue (attempting to 
help the child guess the word by using context clues). 



STUDENT INITIATED QUESTIONS 



This category is used to cover a j>ublic response opportunity that is 
initiated by the student rather than the teacher. Included are situations 
in l^hich the student raises his hand and asks the teacher a question re* 
garding the natter under discussion or some other ouitter. These are similar 
to other response opportunities in that the^ are dyadic teacher-child ia* 
teractions which are public and monitored by the rest of the class. However 
they are not introduced by the teacher and do not involve the child answering 
a question posed by the teacher. These codings are tabulated separately later in 
order to keep them separated from the normal type of response opportunity in 
which the student answers the teacher posed question. 

The student may raise his hand requesting permission to talk, or he 
nay call out his question without permission. If the child calls out, check 
the ^ CALL column. If he is given permission to speak, then leave this col* 
umn blank. 

Relevant (B Is coded if the question has to do with the topic under 
discussion at the time, or if the question has to do with procedures for ac- 
complishing the assignment or activity which is going on at the moment. For 
example, if the class is preparing to do a math assignment, a question about 
the number of problems to do; the procedure for working a particular problem; 
or the time that the assignment is due, would be relevant . 

Irrelevant questions would be any which were not about the current topic. 
If the class was doing a math assignment and a child asked what time school 
dismissed for the day, the question would be coded as irrelevant (IREL). 

Praise (1^) and criticism (^) columns are reserved for coding the teacher's 
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positive or negative evaluation of tha atudentU question. An exanple of 
Praise would be **That*s a good question, Vm glad you asked that/* Criti* 
clsM would be coded if the teacher responded *1hat*s a stupid question. You 
dldn*t think that through.** 

Simple **yes** and **no** answers would not be coded in these columns, they 
should be used only for noting the teacher *s evaluation of the content of the 
ehildU question, not his behavior in asking it. For example, she might 
praise his question, in which case the praise ( \) column would be checked, 
but she might warn him not to call out. This warning should be coded in the 
behavioral warning column. 

The next set of columns provide for recording the type of teacher feedback . 
Wo feedback (0) in coded If she ignores the question and gives no response to 
the child. The teacher may delay her answer to the child because she is do* 
log something else or because she will be answering it in a few moments when 
she is giving directions for doing homework, etc. She may ask him to hold his 
question for later or she may say, **I*11 answer you when I*m through talking 
to Joe.** 

When she does respond to the question she may not accept fNACPT) it into 
the discussion, or otherwise refusm to entertain it. The teacher might say, 
*Ve aren^t talking about what now** or 'let's stick to the subject.** 

Her response may be brief using a few words or a short phrase. 

S: What page are we on? 
T: Page 6. 

A long feedback response from the teacher would involve i\ more detailed 
answer such as: 

S: What page are we on? 

T: Remember, we did the division problems on page S yesterday, so today 
we are going on to page 6 for practice. 
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In the case o£ a redirect s the teacher does not answer the question 
herself^ but directs it to another student or to the vhole class. She aight 
say, "Can anyone answer Johnny?" or ^ell hi«, class." 

The behavioral categories are used primarily for coding those instances 

when the teacher focuses on the child's actions In questioning. These are 

chiefly disciplinary situations where the child has violated sone rule. Ex* 

apples: The teacher tmy entertain the question, but reject the behw^vior. 

She could also praise the behavior. 

Praise: "I like the way Sam got permission before he talked." 

criticism ; "I Just told you the answer to that. If you had been 
paying attention you would knowj" 

Warning : '*Mext time raise your hand and get permission to talk.*' 

STUDENT INITIATED COHMENTS 

Student initiated comments are treated In somewhat the same way as student 
initiated questions. However, since these contributions are comments and not 
questions which require specific answers from the teacher, teacher'd answers 
will be coded differently. Comments may be coded as to whether or not they 
are called out without permission and whether they are relevant or irrelevant 
to the topic under discussion at the time as in the case of student initiated 
questions. 

The teacher may praise iX ^ content of the student's contribution 
by saying, **That*s a good point. We should t«lk about that." She may criticige 
( ) by saying '"That'^ not a good Idea." and thereby negatively evaluate the 
student's comment* 

The teacher may give no feedback (0) at all to the child's comment. This 
situation could occur if the child calls out a comment and the teacher docs not 
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respond to ic. She My delay ber ansver to ttie cobmqC by eaying, *Ve*ll get 

tci that later.'* The teacher my also pot accept (HACPT) by listening to his 

^ — — 

contribution but telling hloi to stick to th:: subject or rejecting his suggestion 
by saying 9 *Ve don't need that nov/* 

The teacher My ali;o accept (ACPT) his coMsent with a nod, an ^'OK" or a 
or in soM other unevaluatlve vay and then tun? her attention to sooeone 
else. In that case the (ACPT) colunn Is ^oded. 

The Integrate colusai Is clt}ecked (INTE6) ^ ) the teacher takes the child's 
cottsaent and Incorporates it Into the class discussion. This could happen if 
for exMpley the class is listing soMthlng like "Rules for Good Hc^alth". 
If the child naaes a good rule and the teacher puts it on the board , the 
coder would code an (IMTEG)* 

Shift is coded it the child's coiment or contribution changes the direction 
of the class dl^cuislon. The teacher may take up the contribution and move 
the discussion along lines dictated by the child's point. 

The behavioral categories are coded for coiiaoencs in the oaM vay as the^ 
are coded for student initiated questions . The teacher My accept and even 
praise the coianentt but vam or criticise the child about calling oct without 
permlsslont or about staying in his seat. 

The coder must be careful not to code as student initiated ccnroents a child' 
answer to a question the teacher asked pre\iously. For exaiqple: 

Teacher: What is this shape? George? 

George: It's a triangle. 

Sam: That's not a triangle. A triangle has three sides. 

Judy: It's a square. 

Teacher: Right, a square. 
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Sam* 8 statement is a student initiated cotmient since it is not an answer to 
a question asked by the teacher, but j.8 merely a comment* Judy*s statement 
is an answer to the teacher's question, however, and is therefore coded as a 
response opportunity rather than a student initiated comment* 



I 
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SELF REFERENCE QUESTKXIS 

The self reference question reqiflres the child to make sone non- 
acadeffllc contribution to the classroom discussion. Any questions which do 
j not involve academic content and/or are not Intended to elicit a particular 

correct factual answer are tallied as self -reference questions. Such questions 
do not have objectively verifiable right or wrong answers. Instead they ask 
the child for his personal experiences, preferences, home life or other factors 
in hl8 personal background* 
Bxamplea: 

Do you have a(dog, car,cold, pencil,)? 
When is your birthday? 

Do you like (arithmetic, ice creau, this story,)? 
What are you doing? 

Bave you ever seen (a football game, the inside of a space capsule)? 
Do you understand the vork? 
Did you do your homework? 

The coder oust determine vhether or ndt the question is subject-matter 
related, that is, whether the question is somehow related to the subject at 
hand* For example, the teacher might begin the introduction of a new unit on 
agriculture with the self question , "Have you ever planted a garden?" A second 
coding decision must be made once subject-matter-related vs. non subject-matter- 
related is determined, and that is whether the question is a request for the 
child to show a preference or to give information about his past experience . 
Once these distinctions are made a hash mark is placed in the appropriate box. 
If the coder cannot decide between subject-matter related and non-subject-mat- 
ter related, then one tally will go in the column marked (?). 
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OPINION QUESTIONS 



Opinion questions occur frequently when the teacher starts a discussion 
on 8ome topic. The teacher's purpose Is usually to get a discussion golng^ 
and her responses to the children are conditioned more by this general aim 
than by a concern for the correctness or Incorrectness of a given opinion. 
Opinion questions require the student to take a position on an issue or to 
predict the outcome of an experiment or hypothetical situation. It assumes 
that the child ^8 opinion stems from an articulated rationale rather than froc 
some chance whim. If pressed he could give reasons as to why he formed it. 
This type of question is usually when the teacher is trying to introduce 
a new unit of study. 

In contrast^ the preference type of self-* reference question previously 

discussed merely asks the child to express a preference or choose among al-* 

ternatives on the basis of taste. The question is not as centrally related 

to curriculum goals as the opinion questio n^ and the child does not have to 

go through an articulated thinking process in order to answer it. 

A few examples of opinion questions would include the following: 
What would you do in that situation? 

Do you think there should be a ivw limiting the number of children 
people have? 

Do you think that people will be living on the moon in the year 20001 
The (NR) No Response column is checked only if the child makes no response 
when asked an opinion i|ue8tion. A response can be verbal or non-verbal. The 
coder may hear or see a child respond, but if the response is not perceived 
by the teacher, it is coded as a No Response (NR). 

The Praise (+) column should be coded if the teacher offers some positive 
evaluation of the content of the child's answer to the opinion question. 
She might Praise by saying, "That's good. *I hadn't thought of that." It is 
also possible for the teacher to Criticize (j^) the child's response or offer 
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one negative evaluation of the child *b response which goes beyond mere 
disagreement. 

Teacher's feedback^ aside from Praise or Criticism^ may also be cod^d 
in the following ways as wetl. She may Ignore (0) the child entirely by turn- 
ing her attention away» responding to another child » or otherwise give no 
feedback of any kind. 

The Disagree column is coded if the teacher has heard the child » but 
indicates that she does not accept what he has said. She may Disagree by 
aaying^ "Oh, I wouldn't do that." "I wouldn't like that." or by offering 
a counter opinion. 

The teacher may also Accept (ACPI) what the child says in some non- 
coomital way by saying "I see.", "OK", or by nodding and indicating that she 
has heard and registered the child's answer. 

Integrates should be checked if the teacher takes the child's opinion 

and weaves it into the ongoing discussion or uses it in any way to build on. 
Example : 

"Bill says that we don't know enough to have people living 
on the moon. This may be true, but what about in the year 
2000?" 
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DYADIC TEACHER-CHILD CONTACTS 

Dyadic teacher- child contacts differ from response opportunities 
and reading and recitation turns in that the teacher is dealing 
privately with one child about matters idiosjmcratic to him rather 
than publicly about material meant for the group or class as a whole. 
The latter distinction is the key one, since teacher-child dyadic 
contacts are not always private (the teacher may talk in a loud voice 
or address the child from across the roonA Such interactions are 
nevertheless coded as teacher-child dyadic contacts as long as they 
involve matters idiosyncratic to the child and are not public ques- 
tions (response opportunities) or reading or recitation turns. 

Dyadic teacher-child contacts are divided into personal, prooecural con- 
tacts, work related contacts, and behavioral or disciplinary contacts. 
They are also separately coded according to whether they are initiated 
by the teacher (teacher-afforded) or by the child (child-created). 
The coding also reflects certain aspects of the teacher's behavior in 
such contacts • * 

All contacts between the teacher and an individual child that do not 
involve reading, recitation or a public response opportunity are coded 
into one of the categories of dyadic contacts (procedural, personal, work-related or 
behavioral). They are separately coded according to whether the teacher or 
the child initiated the interaction. 

Interactions are coded as teacher-afforded if the teacher 
gives feedback about work when the child has not solicited it (the 
teacher either calls the child to come up to his desk or goes around 
the room making individual comments to the students). Created 
contacts are not planned by the teacher and occur solely because the 
child has sought him out; afforded contacts are not planned by t^^e 
child and occur solely because the teacher initiates them. Separate 
space is provided for coding created and afforded work related inter- 
actions on the coding sheets, and the coder indicates the nature of an 
individual dyadic contact by where he codes the interaction. 
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CHILD CREATED CONTACTS 

In dealing with child created contacts, the first necessary decision 
to be made Is whether the contact Is work-related (having to do with either 
content or procedure) or personal (relating to procedure or experience 
charing) . 

(ih I Id-created contacts (work-related) 



There are two types of work-related child-created contacts: content 
related and procedure related. 
Examples: 

1 . content related 

shows work after finishing 
asks for help with problem 
wants to know how to spell a work 
wants to know If answer is right 

2. procedure related ^ 

Asking what page to do, or what problems 
asking permission to read library book 
asking for repetition of assignment 
asking how to title paper 

When a child-created work-related contact ofifcurs, the first decision to make 
Is whether it Is content-related or procedural. Then there are fiiire columns 
divided into two sections In which to record the teacher's feedback to the 
Chi Id. 

I. Evaluative comments (praise and criticism) 

Pra I se (++) should be coded whenever the teacher make a positive 
evaluative comment to the chl Id regarding the quality of his work or the 
effort he Is expending. 

Examples: 

"Your're doing very well. Keep ^t up." 

"I*m very pleased to'see you working so hard." 

"You got all your math problems correct. That's excellent." 



O Praise comments are asually said with feeling and often with some 

ERIC 
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affect such as a smMe, a pat on the shoulder, etc. 
Criticism (-) should be coded when ever the teacher makes a negative 
evaluative comment to the child regarding the quality of his work and the 
effort he Is expending. This negative evaluation goes beyond mere disagreement. 
She may disparage his ability or motlvalTon. 
Examples: 

"You're not trying." 

"I told you to do the exercise on page II. That's page 21." 
"Your papers are always messy. You just don't care." 

Note that nonevaluat Ive comments, those which have In the past been coded 
as "feedback" (FB) are not coded at all. The number of times that the 
teacher gives feedback can be determined by adding the check marks In the 
section next to the praise and critlsism section. This second section will 
always be coded whenever there Is a child-created contact. The praise and 
criticism columns are coded only when they occur. 
2. Extent of teacher feedback to child*created work contacts ; 

The manner in which the teacher gives feedback, aside from evaluative 
conmeots, may be distinguished in any one of the following ways. 

Delay: This column should be coded whenever a student attempts to 
initiate contact with the teacher which is obviously related to work 
(e.g. he approaches the teacher's desk with his workbook, reader, or a 
sheet of paper) and the teacher is occupied or hasn't time at the moment 
to attend to the child and hence, putsthe child off. The teacher may tell 
hin to return to his seat that he (the teacher)will get to hicc later, or 
to wait his turn in line, etc. 

Example: 

A student stands by the teacher's desk with a book in hand. 
The teacher is preparing a note to go to the office. The 
teacher may look up and say, ''I'll get to you in a minute. 
Please sit down/' Or the teacher might simply wave the clkild 
away and point to his chair* 
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Brief should be coded when the Interaction between the teacher and 
child l8 of very short duration. Fpr example, the teacher may glance at 
the workbook the child Is holding and say "GoodJ'' or '•That's flnel". She 
nay respond to a child's question by saying, 'Tage 5." or "In jiour Think 
and Do book." In any case the coder should check brief if the teacher's 
feedback consists of a short sentence (3 or 4 words) or less. 

Examples: 

"Good.'*- would be coded as (-H-) Brief 

"That's terrible!" would be coded as (s:) Brief 

"OK," would be coded as Br.lef only. 
* 

Long is coded whezi the interaction exceeds that of a short sentence 
or phrase, as in the case of Brief . All extended feedback from the teacher 
should be coded in this column* 

Examples: 

"That's good. I'm pleased with your work today." would be 
coded as (-H-) Long . 

"You should have been listening earlier. I told you exactly 
how to work that problem." would be coded as (S) Long . 
"I think you'll find it easier if you use the vocabulary in 
the back of the book." is co^ed Long only. 

The "don't know" (?) category is added for this coding because 
frequently the individual teacher-child interaction that occurs in 
the dyadic contacts will be carried on in hushed tones or across the 
room from the coder where he cannot hear the content of the interaction. 
In such cases, where he is unable to code the nature of the teacher's 
feedback because he cannot hear it, the coder notes the occurrence of 
the interaction and the fact that it was either teacher«>af forded or 
child*created. 

Coders should note that the "don't know" column, has a very 
special and specific meaning for this coding. It should be used only 
when the coder cannot hear the teacher's feedback. It must not be 
used when the coder is unsure about whether to code the teacher's 
teedback as process or product. Thus, use of this column signifies 
that the coder could not hear the interaction, not that he has diffi- 
culty in making a coding decision on the^basis of something that he 
was able to hear. 
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Chlld-'created contacts - Personal 

There are two types of peraonal child-created contacts: 
experience sharing and procedure related* 
1> Experience sharing 

Exatnples: 

Child tells teacher of experience that happened to him 

over the weekend « 
Child tells about event within his family. 
Child tells teacher about not feeling well« 

All experience sharing contacts are personal ones in which the student 

approaches the teacher to tell him something that is not related to 

either classroom work or procedure. 

The teacher's feedback might fall into two categories: 

acknowledge (ACK) or delay (DELAY). The teacher's feedback would 

be coded as acknowledge If the teacher listens to the student's 

experience and perhaps comments on it or simply nods her head and 

acknowledges that she has heard. The teacher's feedback would be 

coded as DELAY if she indicates to the student that she Is unable 

to listen to his experience or talk to him about it at the time. 

2 . Procedure** related 

Examples: 

Procedural interactions created by the child 

Wants paper, pencil, eraser, etc. 

Seeks permission for washroom, drink, etc. 

Finishes work and wants to know what to do 

Has wrong book or worksheet and wants to exchange 

Tattles on other children 

Offers to do a job or errand 

Reminds teacher of something or calls attention to something 

In this situation, where a request for permission Is involved, 
the teacher's feedback may be one of the following: GRANT, (permission is 
given), DELAY (teacher signals the chiJld that she cannot deal with him 
now but will do so later), or NOT GRANT (permission is not given or the 
request is denied). 
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TEACHER A7P0RDED CONTACTS 

Teacher^af forded Contacts (vonrk-related) ; 

The category designations are the same for teacher* afforded 
vork-related contacts as they are for the child-created contacts. The 
same distinctions apply to the praise (-H^), criticise (-) and don't know 
(?) categories for the teacher-afforded situation as for the child- 
created situation. Also, in terms or extent of teacher feedback to a 
given child, the brief and long designations apply here. The one dif« 
ference is that under teach<»r afforded contacts there is added an 
Observes column and the Delay column is omitted. 

Observes is coded whenever the teacher is moving around the room 
glancing at student work, but not entering into verbal interaction. Thus 
It should not be confused with don't know (?) simply because no verbal 
interaction takes place. 
Example : 

The teacher is walking around the room and stops at 
Susan's chair and looks over Susan's shoulder at her 
workbook. The teacher remains here looking for 10 
eeconds or so and then moves to another part of the 
room* 

The coder should check the observe column only when the teacher 
stops and looks at a child's work. It should not be coded if the teacher 
is merely moving around the room scanning aa she moves. Also, if the 
teacher stops and observes but then says something to the child, brief 
or long should be coded and not observe. 

Teacher Afforded Contacts * Personal 

As in the case of child created personal contacts, these contacts 
do not involve either work content or procedure. They aie of a strictly 
personal nature and might involve such things as a teacher asking 
a student about an experience he had on the weekend, about the health 
of some member of the student's family, or perhaps about what happened 
at home the night before to make the child so moody or sleepy. In 
' case of a contact of this sort, a check would be placed In the 
column marked PERS. 
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Te«cher>«f forded contacts (procedure-related) ; 

Within this category a distinction is made between those afforded 
procedures vhich are favors for the teacher, (or those which the child is 
called upon to do which help with the running of the classroom* The child, 
in this case becomes a '*helper'\) and those situations which have to da 
with classroom management or organization* These requests have to do with 
getting the child ready to work on an assignment* 

Bxamples: 

Favor is coded if the^ teacher asks the child to pass out 
the crayolas, workbooks, readers; take a note to the of- 
fice; lead the line to lunch or F«E«; take namee when she 
laaves the room. • 

Hanagement is coded if the teacher aska the child to cover 
his paper, sharpen his pencil, get out his math book, change 
his seat in the classroom* 

Thar^^ yoti is checked if, in addition to an afforded procedure, the 
teacher thanks the child for performing the favor * Thank you*8 will be 
heard more frequently in connection with the teacher's request for a 
favor from the child than in the management situation, however, it would 
not be impossible for them to occur following management requests* 

Examples: 

T: Laura, will you pass out the lunch cards, please* 

(code teacher-afforded procedure, favor) 
S: (Passes out cards and sits down) 
T: Thank you, Laura. (Check thank you column*) 

T: John, get out a clean sheet of paper* (Teacher-afforded 

procedure, management) 
S: (John gets out paper*) 

T: (Teacher begins writing on board and turns attention away 
from John* ) (No thank you is coded) 



Behavioral Contacts 

Behavioral contacts are coded whenever the teacher makes sane 
comment upon the child's classroom behavior* They are subdivided 
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Into praise , non-verbal Intervention , warnings . and criticism . 

Behavioral evaluation contacts are considered to be teacher 
afforded , although they usually occur as reactions to the child's 
ijomediate ly preceding behavior. Nevertheless, they are teacher-* 
afforded in the sense that the child usually does not want and does 
not expect the interaction, and the teacher chooses to single the child 
out for connent. The conditions for coding this category are: 
(a) the teacher singles out the child for comnent upon his classroom 
behavior; (b) the interaction concerns only his behavior and does not 
Involve praise or criticism in connection with work-related or procedural 
contacts as defined above. Some behavioral criticism may occur in 
vork--related and procedural contacts, and in those situations it appears 
in the coding for work-related and procedural interactions. The 
category of behavioral interactions is used only for those instances 
in which the teacher singles out th^ child for comment solely on the 
basis of wanting to discuss his classroom behavior. Work-related or 
procedural matters are not involved. 

Most of the evalua* 
tlons coded in this category will occur in connection with the child's 
attention, cooperation, and performance of classroom rituals, although 
occasionally they will be comments made in relation to the child's academic 
work. In the latter case, there will be evaluations made at the conclusion 
of a lesson or a scl.ool day in which the teacher refers to the child's 
general performance. Teacher praise or criticism of this sort would not 
be picked up by the coding system otherwise, since it does not occur as 
part of a response opportunity, reading or recitation turn, and other 
dyadic contact. 

Praise 

This category will be used relatively infrequently with most 
teachers, although it will occur. Occasionally children will be 
singled out for special praise when they have done a particularly 
good Job of cleaning up their desks, sitting up straight, keeping 
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quiet In preparation for leaving the room, etc. Praise coded in this 
category will also sometimes occui^ after activities but not in rela* 
tion to specific responses during those activities ("Johnny really 
knew all his words today — he must have studied real hard last 
night.*') • Idiosyncratic teacher euphemisms that carry the same sorts 
of meanings as the preceding examples are also considered to be praise 
(^'Johnny has on his listening ears today,*' *'Mary knows how to get 
ready to go/'). Whenever the teacher singles out a child for such 
praise 9 coders should check the praise (4-f) column under behavioral 
teacher-afforded contacts > 

ExaaiplAs: 

1. Praise 

"John is ell ready/' (has hisfasnds folded, is sitting up, etc.) 
"John's got his listening ears on today." 

"John^ you really knev your words today, didn't you?" (said after the 
lesson rather than during a response opportunity) 

Non^'Verbal Intervention is included in this system to account for those 
situations in which the teacher takes steeps to correct a behavioral prob- 
lem» however, she does so vithoi^t disrupting the whole class. She may 
move close to a child who is talking; she may tap a child on the shoulder 
who is daydreaming and point to his book; or she could turn a child around 
in his seat when he is facing the wrong way and looking at his neighbor. 
These are cases where the teacher does intervene, but does so inaudibly 
with a minioMim of disruption. 

Wamin j ^ 

This category and the following one refer to teacher behavior in 
singling out for comment a child engaging in inappropriate or undesirable 
classroom behavior. Comments and audible gestures ^ such as tapping a ruler 
on the desk or finger snapping, which function as warnings and which do 
not include elements codable as criticism «re coded in the warning category, 
while negative reactions which do contain criticism are coded in the 
criticism category to be described below. Usually teachers* warnings 
will occur in situations in which the child is doing something that is 
not necessarily or always prohibited but which is troublesome iat the 
moment. In such instances the teacher till single out the child to 
inform him that his present behavior is inappropriate, but will do so 
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without couBUnicatlon cf rejection or anger mu in criticism. Examples 
of thia are as follows: 'UohnnyH you •re getting too noisy" "Try to figure 
out the answer on your own <bn*t copy off your neighbor" "Johnny, you 
can talk to Mary if you want to, but stay in your seat." 

The lines of demarcation between procedural-afforilsd interactions 
and behavioral warnings, and between behavioral waniin^s and behavioral 
criticisms, are sometimes difficult to discern. 

Sometimes the same or nearly the same words couid be 
coded in either category, with the decision being made cn the basis of 
the nonverbal expressiv^s and gestural components of the teacher's 
message. Behavioral instructions given to the child merely in the in- 
terest of information or classroom management and without any connota- 
tion of warning or criticism would bo coded as afforded procedural con- 
tacts. The same instructions given in a slightly different context 
which connoted more of a warning and perhaps implied that Che child 
should know better ("John sit down — Mary can't see whea you stand up 
like that.") would be coded as behavioral warnings , if the same S3ntence 
were snapped at the child or delivered* with anger or exasperation, it 
would be coded as behavioral criticism. 

Warning 

'*You're too louo, John." 

''Stay in your seat, John." 

"Raise your hand if you want to answer." 

"Try to figure out the answers yourself." 

Teacher snaps her fingers at a child who is not paying attention. 

3. Criticism 

"Keep your voice down, John!" (with irritation) 
"John — sit down!!" 

"I told you to raise your hand first don't you listen?" 

"Keep your eyes to yourself, John, his paper is none of your business" 
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BEHAVIOR*REIATED CONTACT ERRORS 

When coding a desist event (the stopping by the teacher of misbehavior), 
we would like to obtain a measure of her effectiveness of method « We can do 
this by recording certain errors which she may make when halting a deviancy, 
target, timing, overreaction, and shift errors. 

A TARGET ERROR is coded when the desists the wrong student or desists an 
onlooker or contagee rather than an initiator. For example, all is quiet until 
Mary whispers to Jane. Jane thei} says something back to Mary, and Jami turns 
around to listen. "Jami, turn around and gee back to work," the teacher says. 
A target error is also coded if one deviancv is stopped while another, more 
serious misbehavior was allowed to continue. Thus, if Bob were tossing paper 
airplanes while the teacher was chastizing Mary and Jane, that would be a target 
error. 

A TIMING ERROR is coded whenever misbehavior increases in seriousness or 
spreads to more children before being halted. For instance. Jack whispers to 
Craig, who whispers to Jim, and then Barney whispers to Craig, and then the 
teacher desists. Also, if John says something to Clem, Clem pokes John, John 
pokes Clem, and they start to pull each other's shirts off before the teacher 
stops them, the desist is considered "too late" because the misbehavior increased 
in seriousness before she acted. 

Occasionally the coder will be busy coding other information prior to a 
desist and will not have been able to gather sufficient evidence to judge whether 
or not a target or timing error has been made. In these cases, place a check in 
the "?" column. xTiis refers only to the target and timing error columns, since 
the coder can usually tell if an overreaction or shift error has been made 
without having previously observed the children.. 
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An over reaction error occurs whenever a teacher overreacts to a devlancy. 
For exanqile, if Mary and Jane stop talking and get back to work before the teacher 
desists, OVRCT should be coded; since the misbehavior had already stopped, the 
teacher should have Ignored it. It would not be a timing error because the mis- 
behavior does not spread or become more serious. Another Instance: The class 
is in a discussion and Hercules is talking when tlie teacher says, "Hercules, stop 
talking. This is not a playground, it's a classroom, and we're supposed to be 
working. If you talk, you disturb your neighbors so they can't work. So let's 
all get back to work and be quiet." This overdwelling on the point is an over- 
reaction error because the teacher's action is more than sufficient to stop the 
talking. Of course, if a serious devlancy such as a fight occurs, stem action 
would be appropriate since the class has already been disturbed. Even so, the 
teacher can ccrmnit an overreactlon error by criticizing the deviants b^ond the 
point where they understand and conform. * 

The ••NOERR" column is checked whenever the teacher desists without committing 
any of the four errors. As mentioned before, the '*?" should be used and the 
•'NOERR" column not checked if the coder is not sure that a target or timing error 
has not occurred. 
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Appendix B 




CURRICULUM AREA METHODS ^ND MATERIALS 



MECHANICS OF CODING: 

This sheet is designed to be continuously coded during curriculum 

related activities such as reading, math, spelling, science, etc. There 

are three types of categories (A, Teaching, B, Methods, and C. Materials), 

each with eight to ten subcategories which are defined in the next 

section. In addition, an entry should be made to indicate the degree 

of individualization, occording to the following code: 

I - teacher behavior is directed toward an individual, 
she is requesting individual work or responses, or 
materials are individually tailored for the various 
individual children 

P - teacher behavior is directed at a subset of the total 
instructional group, or she is having the children 
work in pairs or small groups 

G - teacher behavior is directed toward the entire group 

she is working with, or tho group response is chorused, 
or the group is working together as a whole on some 
activity; for materials, use this category whon all 
children have the same materials even though each 
child has his own copy 

Time coded usually refers to category A. When ^8 in this 
category (follow-cp activity) is done as seatwork it will not be timed, 
but should be listed at the end of the lesson with as many subcategories 
from C as apply. When the follow-up activity is done in the group, 
enter the time in the same way as for other A categories. Sometimes 
you will need to enter the time to show a change in methods (some B 
category) even though the A category has not changed; if so, repeat 
the A category entry and mark the appropriate B and C entries. 

There will not always be entries in column C as physical materials 
may not be used. For example, if the' teacher is just talking to 
explain directions for an activity, the B entry would be //2 and there 
would be no C entry. 

If the lesson is long and varied you may need to go on to a 
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Curriculum area methods and materials - pt 2 

second sheet. Enter the stop time of the lesson at the end, and 
begin each new lesson or group session with a new sheet. 

The I-P-G dimension may be different for the different categories 
on the same line. For instance, if the teacher is working with the 

whole group, calling on children to recite individually, and each child 

3 

has a copy of the same book, the A entry would be 2G, the B entry would 

be 51, and the C entry would be IG. 

Elapsed time can be figured and entered after the total observation 

session is over. A separate sheet will be used to total the various 

subcategories. You may need to start a new line and make new entries 
when less than a minute has passed. This will be corrected when elapsed 
DEFINITIONS OF THE CATEGORIES: time it figured. 

General information in the heading - Always fill in the complete 

heading at the top of the page. For area and purpose of activity enter 

the subject matter area and, if there is occasion, the special purpose 

of the lesson (ex: arithmetic, number facts drill). Group size refers 

to the number of children in the group. By the end of the session 

you should be able to judge the rank out of the number of groups you 

saw. 

A> Teaching categories - generally refers to ports of a lesson. 

1. Review of old material refers to going over materials which you 
have not seen taught to the group. You may find this at several 
points in the lesson (ex: in a reading group when old vocabulary 
is reviewed before new is presented, and then again later in a new 
phase of the lesson when there is discussion of what was read 
yesterday before beginning to read new material today). Some 
teachers may never use this category at all. 

2. Presentation of material is the actual teaching of something new. 
The teacher may cue this category for you by referring to moving on 



Curriculum area methods and materials - p. 3 

to something new. Other times you will have to infer it from the 
behovior of the children (ex: only a few children able to respond 
correctly, slow or stumbling responses, puzzlement followed by 
"ohal" responses). Include in this category the first time going 
through a reading story. 

3# Practice refers to still-«in<»the-group supervised practice with 
concepts or materials you have just seen presented. It begins at 
the point where the teacher appears to have made the judgment that 
most of the kids have gotten the point end so now she begins to 
have them practice it. In reading the teacher may stop telling what 
syllable is accented and begin to drill the children on doing this, 
or the group may go back to the beginning of the day's reading 
selection and re-read for fluency or detail. In math the students 
may work oral examples of the principle presented. Teacher 
explanation should be minimal, only for occasional errors or a few 
of the children. If the practice breaks down because the children 
don't really understand and the teacher begins to re<»present the 
material to the whole group, go back to the presentation of material 
category (#2). 

WHEN IN DOUBT BETWEEN #1, #2, and #3, use #2. 

^« Summorizing review refers to situations in which the teacher asks 
questions or gives comments which in some way synthesize or help 
the student to organize the content of the lesson. For example, 
if the reading lesson included work on the effect of a silent "e," 
at the end of the lesson the teacher may ask one child, or the 
group in chorus, to give her the rule for short at the end of 
o word. Another situation would be listing the four things the 
fox did in the story, or three rules for writing equations^ Some- 
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times the teacher may simply tell or ask a student to tell what 
the lesson was about today. This may not be a frequently occurring 
category* 

5. Teacher evaluation is a general performance evoluotion given by the 
teocher. Do not attempt to code brief statements of "Goodl" or 
"Nice vrorkl" but do include more definite periods of this type of 
evaluation » as well as any time the teacher spends in the group 
correcting the children's work, 

6. Student self^^evaluation refers to those student self-evaluations 
elicited by teacher prompting. In oxher words » she is intentionally 
trying to get them to develop their own skills of self •-evaluation. 
It includes when the i^^^ache^ has the children correct their own 
written work. Oo not use this category for cases in which the 
students spontaneously offer self comments. 

7. rollow^'Up instructions refers to presentation of and directives 
pertaining to any follow-up work (soatworkt homeworkt activities) 
which will give additional practice on the material you saw taught 
in the group session* but will be done independently by the 
children. 

8. Follow-up activity is the independent practice of material or 
concepts taught in the group session* (Note: Do not confuse 
independently with individually* which applies to the I-P-G column. 
The latter would mean that the materials are individualized.) 
There will be no time code for this categoryt simply a tally mark 
for as many column C materials as are applicable, if the students 
leave the group to do the work. If they stay at the group orea 
and either correct their work or turn it in to be corrected, enter 
th<? time in the same manner as for other A categories. 
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The I-P-6 column for the A category should always reflect whether 
the teacher's attention is directed toward the whole group, or if she 
sets some of the children to working independently so that she can 
work more individually with a subset of the group (p) or an individual (l) 
Watch for times when she sends the group back to their seats to work 
independently and retains one or more children for more help. 
B. Methods 

Demonstnation. diagram includes situations when the teacher -models, - 
that is, she demonstrates or illustrates what is to be done. For 
example, she may read a sentence or paragraph or may work a sample 
problem on the board. Very occasionally you may need to use this 
category for a child's recitation turn, if the teacher specifically 
has a child demonstrate (-John, you show how it should be done.-), 
but do not use it for an ordinary practice turn or when o second 
child gives a correct answer which another child has missed. 
2. Lecture is when the teacher is just talking, giving information or 

instructions, with little illustration or example. 
3.,& 4. One of the two discussion categories should be used whenever 
the group is talking about a story, problems, etc., without having 
to decide upon correct answers. This includes situations where the 
children are correcting (evaluating #6 in A) their own work together 
in the group. Focussed discussion involves the teacher using 
prepared, sequenced questions to help students arrive at conclusions, 
or to understand verbal material. Unfocussed discussion refers to 
rambling conversation, without any apparent objective, perhaps 
- dominated by student digressions. 
Pupil rea ding/recitation is when the children take turns reading. 
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doing moth problems, reading the filmstrip titles, etc., with the 
intention that the child's performance is primarily a learning 
experience for himself rather than on example for the other children. 
This will probably be the most frequent category in which the "1" 
is used. If the reading is silent, indicate that in parentheses: 
51 (silent). 

6- Pi-j-ll should be used when the practice is aimed at speed and 
automaticness without any particular effort to think about the 
situation. 

Problem solving is process oriented practice, most probably some type 
of opportunity to transfer or appl> what has been learned. This 

category should be used with workbooks, problems on the board, etc. 

any thinking situations in which the children ore to come up with 
answers. (But remember that correcting problems previously done is 
considered discussion, #3 or #4.) 
8. Dead spots refer to situations in which the flow of the group's 
activity is broken because the teacher attends to something else 
(looks up something in the manual, gets materials, is interrupted 
by a student in the class, disciplines someone outside the group, 
or within it if this is disruptive of the group). Do not count 
episodes initiated from outside the classroom (PA announcements, 
messengers, etc.). 
9' & 10. Patterned turns and non«patterned turns indicate if the 
teacher colls on the children in any recognizable sequence (so 
that they con safely "not think" for some turns or questions). 
These two categories will always be double coded with numbers 3 
through 7 in column B. 
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The I«»P*6 dimension for column B can usually be decided by whether 
the children have individual turns or the whole group is responding 
jointly. Discussion, however, would be group if questions ere directed 
generally and everyone has a chance to talk, even though only one 
person talks at a time. 

There will often be double codings in column B« For example, the 
group may take turns reading workbook selections and telling the 
answer, with the children each reading in turn going around the group: 
this would be coded 7G, 51, 9, all in column B, In these types of 
situations, make entries on separate lines of column B« You do not 
need to repeat the A entry, but be sure to start the next time entry on 
a totally new line. Please always moke a decision between discussion 
(talking cbout) and problem solving (thinking of an answer): do not 
double code #3 and //7 in the B column, §5 may be double coded with §7^ 
if tho children ore taking, turns in the problem solving, but do not 
double code §5 with discussion {§3 or ^4) or drill (jj'6), 
C. Materials 

1, Standardized refers to published* pre«»packaged, purchasible mattsrials 
including using the exercises directly out of the teacher^s manual, 

2, Teacher^created refers to materials specifically designed for the 
group by the teacher; it is usually distinguishable by being hand«> 
written or handmade, 

3, Seatwork/homowork refers to specifically assigned independent work, 
to be checked at some point by the teacher, 

^' A/V aids ; overhead projector, films, records, filmstrips, records, e 
5, Games/activities may or may not be checked later by the teacher, 
but arc distinguishable as being more ''fun" thon assignments 
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I 6, Leorning centers refer to areas in the room set up for specific 
activities f where the children can do something someplace other 
than at their own desks. 
?• Excursion , either planned or mentioned, 

8, Free time; no specific directions are given, or simply ••do 
I something when you are finished,*^ 

The I-P-G dimension for category C should refer to whether the 
I materials usea by the children ore differentiated according to their 
j level or interests. If every one hos his own copy of the book or 

materials, but every child has the same thing, code G. If they are 
I reading or ploying games in pairs or small groups, code P. If each 

one reads a different library book or uses a different instructional 
i game, code I. 

I Column C may also often involve double coding, especially of 

categories #1 and //2 with the others. As with column B, use as many 

j lines as necessary to indicate all the materials involved, but be 
sure to start the next time entry on a completely new line. 



Note: Sometimes when the children in a group have be^n set to work 
independently on a task (SA). the teacher will give instructions or 
help to individuals (either within the group or in the rest of the 
class). Since the teacher does not really take o*^t • the group again, 
nor does sho really leave it. such teacher behavior should be coded 
as ••intermittent'* in the Start Time column, entries mode in the 
regular way in the A.B. and C columns, and time spent entered in the 
Elapsed Time column. . 

A I B i c i 
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I During a lesson listening on earphones to records, the 

te jchor gives one student additional instruction?, . and 
scolds another. 

"T" 



Schoo yritAMsi/rJ Teache r A>&^ Gr, Date ^^>C30 Obsorvo r>l/tM4 P. / of A 



ea C. purpose of octivit y 1(»^A.A 



^. Teaching CQtogorios 

Il.Revicw of old materiol 
2«rres3ntation of moteriol 
3*Practic3 

f4.Sunr(X4rizing review 
S.TsQchor evcluoticn 
6«Studcnt self-^evoluQtion 
J 7 •Follow-up instructions 
I 8«Follow«-up activity 

B, Methods catogorios 

!^ l.DorAonst ration, diagram 
a.Lecturo 

3«FocMSsed discussion 
r 4«?Mfocussod discussion 
j S.Pgpil reading /reel tot ion 

^•Orill 

T.Problem soJving 
I 6«Otod spots . 
* ?. Patterned turns 
iO.Non«»pattemed turns 



f%terlpls categories 

l.Stondordized 
i« Teacher created 
3 • Seatwork/homev^ork 
4.A/V oids 
5 • Gomes/oet i v i trl es 
6«l.eornlng centers 
7«Excursicn 
8 .Free tifwe 



Group size T. rank^ 
Codes V 
B 



2. 
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1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 

11. i 

12. 
13.*~ 
14. 
).'5. 
16. 
17. 
If.' 
19 

at.*" 

21.' 
so. 

31 r 

33. 
34."" 
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Example ; At 8:30 the teacher begins a reoding group by dt illing the 
entire group with flashcards she has made up with previous vocabulcsry 
words, calling on one child at a time in a non->patterned order. At 
8:34 she begins to introduce new vocabulary, writing the words cn the 
board and letting the children call out the words as they recognize 
them. At 8:38 the children begin to read o new story* reading a 
paragraph individually in turns going around the group. Two children 
have particular difficulty, so when they finish one time reeding it 
through (8:47) the teacher divides the group, listening to the two 
poorer readers while the rest of the group reads it through without 
direct supervision from her. When this is finished (8:53) thr teocher 
compliments the group on their performance this morning, briefly gives 
directions for workbook pages, and sends the children bock to their 
seats to do the assignment. 



Curriculum Area Methods and Materials 
Directions for Tallying 

PREPARING THE CODING SHEETS: 

1. Go through all coding sheets and be sure there are no entries 
for time spent taking tests or when c student teacher wos 
teaching. If there are such entries cross them out. 

2. Figure total time coded by subtracting beginning time from 
ending time for each lesson ("Start time" column) and summing 
for all lessons for that teacher. (Be sure to subtract the 
amount of time consumed by tests and student teachers.) 

3- Fill in the elapsed time column. If there is more than one 

entry for the same minute of start time, split the time in the 
elapsed column (e.g., two entries at 8:41 would each be counted 
as 30 seconds elapsed time; three entries would be counted as 
20 seconds each). Elapsed time should equal total time coded. 

4. To help handle the scoring of double coding, be sure that any 
double coding within the A-B-C columns is bracketed. Circle 
any periods coded as "intermittant . " 

TALLYING: 

5. Enter each unit of elapsed time in the proper category of A-B-C/ 
I-P-G (except B8, B9, and BIO, which have no I-P-G designation). 
Enter B5 and B5s (silent reading) separately. For non-supervised 
independent activity (A8 not timed), tally the materials used 

on the bottom C line. 

6. Count the number of entries for each category, write the total 
above the line at the top of the category (line labelled ••f'') 
and sum across the I-P-G categories for a grand total frequency 
for each numbered category. 

7. Total the time entries for each column at the bottom. Sum across 
the I-P-G categories for a grand total of time for each numbered 
category. 

8. From the numbers at the bottom of each column (excluding B8, B9, 
and BIO) total the I entries, the P entries, and the G entries 
for A, for B, and for C. 

9. Sum the time total for each numbered category for A, for B, and 
for C to get total lesson time (A), total teaching time (b), 
and total materials time (C); these times will differ from 
total time coded (stop ^2) because of double coding or no coding. 
For A and C this time should equal the sum of the I-P-G totals; 
for B it should equal the sum of the I-P-G totals plus categories 
8, 9, and 10. 

10. Sum the I-P-G across A-B-C to get total I-P-G. 

11. To figure effects of double coding, go through the coding sheets 
and figure the amount of A entries, B entries, and C entries which 
are not coded, are single coded, and are multiple coded. Within 
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A, B, and C the total of no entry + single coding + multiple 
coding should always equal total time coded. 



Example: 




In this case the one minute of A6I and two minutes of 
A71 are double coding, Tho B3I and 821 entries are single 
coded. There are seven minutes of single coded A and three 
minutes of multiple coded A; seven minutes of no B and 
three minutes of single coded B; ten minutes of multiple 
coded C, Total time coded is ten minutes, (Lesson time 
is thirteen minutes; teaching time is three minutes; 
materials time is twenty minutes,) 



